Context Navigation

em86real.S @ ba46ffa6

4.104.114.84.95

Last change on this file since ba46ffa6 was ba46ffa6, checked in by Joel Sherrill <joel.sherrill@…>, on 06/14/99 at 16:51:13

This is a large patch from Eric Valette <valette@…> that was
described in the message following this paragraph. This patch also includes
a mcp750 BSP.

From valette@… Mon Jun 14 10:03:08 1999
Date: Tue, 18 May 1999 01:30:14 +0200 (CEST)
From: VALETTE Eric <valette@…>
To: joel@…
Cc: raguet@…, rtems-snapshots@…, valette@…
Subject: Questions/Suggestion? regarding RTEMS PowerPC code (long)

Dear knowledgeable RTEMS powerpc users,

As some of you may know, I'm currently finalizing a port
of RTEMS on a MCP750 Motorola board. I have done most
of it but have some questions to ask before submitting
the port.

In order to understand some of the changes I have made
or would like to make, maybe it is worth describing the
MCP750 Motorola board.

the MCP750 is a COMPACT PCI powerpc board with :

1) a MPC750 233 MHz processor,
2) a raven bus bridge/PCI controller that
implement an OPENPIC compliant interrupt controller,
3) a VIA 82C586 PCI/ISA bridge that offers a PC
compliant IO for keyboard, serial line, IDE, and
the well known PC 8259 cascaded PIC interrupt
architecture model,
4) a DEC 21140 Ethernet controller,
5) the PPCBUG Motorola firmware in flash,
6) A DEC PCI bridge,

This architecture is common to most Motorola 60x/7xx
board except that :

1) on VME board, the DEC PCI bridge is replaced by
a VME chipset,
2) the VIA 82C586 PCI/ISA bridge is replaced by
another bridge that is almost fully compatible
with the via bridge...

So the port should be a rather close basis for many
60x/7xx motorola board...

On this board, I already have ported Linux 2.2.3 and
use it both as a development and target board.

Now the questions/suggestions I have :

1) EXCEPTION CODE

As far as I know exceptions on PPC are handled like
interrupts. I dislike this very much as :

a) Except for the decrementer exception (and
maybe some other on mpc8xx), exceptions are
not recoverable and the handler just need to print
the full context and go to the firmware or debugger...
b) The interrupt switch is only necessary for the
decrementer and external interrupt (at least on
6xx,7xx).
c) The full context for exception is never saved and
thus cannot be used by debugger... I do understand
the most important for interrupts low level code
is to save the minimal context enabling to call C
code for performance reasons. On non recoverable
exception on the other hand, the most important is
to save the maximum information concerning proc status
in order to analyze the reason of the fault. At
least we will need this in order to implement the
port of RGDB on PPC

==> I wrote an API for connecting raw exceptions (and thus
raw interrupts) for mpc750. It should be valid for most
powerpc processors... I hope to find a way to make this coexist
with actual code layout. The code is actually located
in lib/libcpu/powerpc/mpc750 and is thus optional
(provided I write my own version of exec/score/cpu/powerpc/cpu.c ...)

See remark about files/directory layout organization in 4)

2) Current Implementation of ISR low level code

I do not understand why the MSR EE flags is cleared
again in exec/score/cpu/powerpc/irq_stubs.S

#if (PPC_USE_SPRG)

mfmsr r5
mfspr r6, sprg2

#else

lwz r6,msr_initial(r11)
lis r5,~PPC_MSR_DISABLE_MASK@ha
ori r5,r5,~PPC_MSR_DISABLE_MASK@l
and r6,r6,r5
mfmsr r5

#endif

Reading the doc, when a decrementer interrupt or an
external interrupt is active, the MSR EE flag is already
cleared. BTW if exception/interrupt could occur, it would
trash SRR0 and SRR1. In fact the code may be useful to set
MSR[RI] that re-enables exception processing. BTW I will need
to set other value in MSR to handle interrupts :

a) I want the MSR[IR] and MSR[DR] to be set for
performance reasons and also because I need DBAT
support to have access to PCI memory space as the
interrupt controller is in the PCI space.

Reading the code, I see others have the same kind of request :

/* SCE 980217

*

We need address translation ON when we call our ISR routine

mtmsr r5

*/

This is just another prof that even the lowest level
IRQ code is fundamentally board dependent and
not simply processor dependent especially when
the processor use external interrupt controller
because it has a single interrupt request line...

Note that if you look at the PPC code high level interrupt
handling code, as the "set_vector" routine that really connects
the interrupt is in the BSP/startup/genpvec.c,
the fact that IRQ handling is BSP specific is DE-FACTO
acknowledged.

I know I have already expressed this and understand that this
would require some heavy change in the code but believe
me you will reach a point where you will not be able
to find a compatible while optimum implementation for low level
interrupt handling code...) In my case this is already true...

So please consider removing low level IRQ handling from
exec/score/cpu/* and only let there exception handling code...
Exceptions are usually only processor dependent and do
not depend on external hardware mechanism to be masked or
acknowledged or re-enabled (there are probably exception but ...)

I have already done this for pc386 bsp but need to make it again.
This time I will even propose an API.

3) R2/R13 manipulation for EABI implementation

I do not understand the handling of r2 and r13 in the
EABI case. The specification for r2 says pointer to sdata2,
sbss2 section => constant. However I do not see -ffixed-r2
passed to any compilation system in make/custom/*
(for info linux does this on PPC).

So either this is a default compiler option when choosing
powerpc-rtems and thus we do not need to do anything with
this register as all the code is compiled with this compiler
and linked together OR this register may be used by rtems code
and then we do not need any special initialization or
handling.

The specification for r13 says pointer to the small data
area. r13 argumentation is the same except that as far
as I know the usage of the small data area requires
specific compiler support so that access to variables is
compiled via loading the LSB in a register and then
using r13 to get full address... It is like a small
memory model and it was present in IBM C compilers.

=> I propose to suppress any specific code for r2 and
r13 in the EABI case.

4) Code layout organization (yes again :-))

I think there are a number of design flaws in the way
the code is for ppc organized and I will try to point them out.
I have been beaten by this again on this new port, and
was beaten last year while modifying code for pc386.

a) exec/score/cpu/* vs lib/libcpu/cpu/*.

I think that too many things are put in exec/score/cpu that
have nothing to do with RTEMS internals but are rather
related to CPU feature.

This include at least :

a) registers access routine (e.g GET_MSR_Value),
b) interrupt masking/unmasking routines,
c) cache_mngt_routine,
d) mmu_mngt_routine,
e) Routines to connect the raw_exception, raw_interrupt
handler,

b) lib/libcpu/cpu/powerpc/*

With a processor family as exuberant as the powerpc family,
and their well known subtle differences (604 vs 750) or
unfortunately majors (8xx vs 60x) the directory structure
is fine (except maybe the names that are not homogeneous)

powerpc

ppc421 mpc821 ...

I only needed to add mpc750. But the fact that libcpu.a was not
produced was a pain and the fact that this organization may
duplicates code is also problematic.

So, except if the support of automake provides a better solution
I would like to propose something like this :

powerpc

mpc421 mpc821 ... mpc750 shared wrapup

with the following rules :

a) "shared" would act as a source container for sources that may
be shared among processors. Needed files would be compiled inside
the processor specific directory using the vpath Makefile
mechanism. "shared" may also contain compilation code
for routine that are really shared and not worth to inline...
(did not found many things so far as registers access routine
ARE WORTH INLINING)... In the case something is compiled there,
it should create libcpushared.a

b) layout under processor specific directory is free provided
that

1)the result of the compilation process exports :

libcpu/powerpc/"PROC"/*.h in $(PROJECT_INCLUDE)/libcpu

2) each processor specific directory creates
a library called libcpuspecific.a

Note that this organization enables to have a file that
is nearly the same than in shared but that must differ
because of processor differences...

c) "wrapup" should create libcpu.a using libcpushared.a
libcpuspecific.a and export it $(PROJECT_INCLUDE)/libcpu

The only thing I have no ideal solution is the way to put shared
definitions in "shared" and only processor specific definition
in "proc". To give a concrete example, most MSR bit definition
are shared among PPC processors and only some differs. if we create
a single msr.h in shared it will have ifdef. If in msr.h we
include libcpu/msr_c.h we will need to have it in each prowerpc
specific directory (even empty). Opinions are welcomed ...

Note that a similar mechanism exist in libbsp/i386 that also
contains a shared directory that is used by several bsp
like pc386 and i386ex and a similar wrapup mechanism...

NB: I have done this for mpc750 and other processors could just use
similar Makefiles...

c) The exec/score/cpu/powerpc directory layout.

I think the directory layout should be the same than the
libcpu/powerpc. As it is not, there are a lot of ifdefs
inside the code... And of course low level interrupt handling
code should be removed...

Besides that I do not understand why

1) things are compiled in the wrap directory,
2) some includes are moved to rtems/score,

I think the "preinstall" mechanism enables to put
everything in the current directory (or better in a per processor
directory),

5) Interrupt handling API

Again :-). But I think that using all the features the PIC
offers is a MUST for RT system. I already explained in the
prologue of this (long and probably boring) mail that the MCP750
boards offers an OPENPIC compliant architecture and that
the VIA 82586 PCI/ISA bridge offers a PC compatible IO and
PIC mapping. Here is a logical view of the RAVEN/VIA 82586
interrupt mapping :

| OPEN | <-----|8259|
| PIC | | | 2 ------
|(RAVEN)| | | <-----|8259|
| | | | | | 11
| | | | | | <----
| | | | | |
| | | | | |

^{------

| VIA PCI/ISA bridge

| x

-------- PCI interrupts}

OPENPIC offers interrupt priorities among PCI interrupts
and interrupt selective masking. The 8259 offers the same kind
of feature. With actual powerpc interrupt code :

1) there is no way to specify priorities among
interrupts handler. This is REALLY a bad thing.
For me it is as importnat as having priorities
for threads...
2) for my implementation, each ISR should
contain the code that acknowledge the RAVEN
and 8259 cascade, modify interrupt mask on both
chips, and reenable interrupt at processor level,
..., restore then on interrupt return,.... This code
is actually similar to code located in some
genpvec.c powerpc files,
3) I must update _ISR_Nesting_level because
irq.inl use it...
4) the libchip code connects the ISR via set_vector
but the libchip handler code does not contain any code to
manipulate external interrupt controller hardware
in order to acknoledge the interrupt or re-enable
them (except for the target hardware of course)
So this code is broken unless set_vector adds an
additionnal prologue/epilogue before calling/returning
from in order to acknoledge/mask the raven and the
8259 PICS... => Anyway already EACH BSP MUST REWRITE
PART OF INTERRUPT HANDLING CODE TO CORRECTLY IMPLEMENT
SET_VECTOR.

I would rather offer an API similar to the one provided
in libbsp/i386/shared/irq/irq.h so that :

1) Once the driver supplied methods is called the
only things the ISR has to do is to worry about the
external hardware that triggered the interrupt.
Everything on openpic/VIA/processor would have been
done by the low levels (same things as set-vector)
2) The caller will need to supply the on/off/isOn
routine that are fundamental to correctly implements
debuggers/performance monitoring is a portable way
3) A globally configurable interrupt priorities
mechanism...

I have nothing against providing a compatible
set_vector just to make libchip happy but
as I have already explained in other
mails (months ago), I really think that the ISR
connection should be handled by the BSP and that no
code containing irq connection should exist the
rtems generic layers... Thus I really dislike
libchip on this aspect because in a long term
it will force to adopt the less reach API
for interrupt handling that exists (set_vector).

Additional note : I think the _ISR_Is_in_progress()
inline routine should be :

1) Put in a processor specific section,
2) Should not rely on a global variable,

As :

a) on symmetric MP, there is one interrupt level
per CPU,
b) On processor that have an ISP (e,g 68040),
this variable is useless (MSR bit testing could
be used)
c) On PPC, instead of using the address of the
variable via CPU_IRQ_info.Nest_level a dedicated
SPR could be used.

NOTE: most of this is also true for _Thread_Dispatch_disable_level

END NOTE

Please do not take what I said in the mail as a criticism for
anyone who submitted ppc code. Any code present helped me
a lot understanding PPC behavior. I just wanted by this
mail to :

1) try to better understand the actual code,
2) propose concrete ways of enhancing current code
by providing an alternative implementation for MCP750. I
will make my best effort to try to brake nothing but this
is actually hard due to the file layout organisation.
3) make understandable some changes I will probably make
if joel let me do them :-)

Any comments/objections are welcomed as usual.

/ ` Eric Valette

/-- o _. Canon CRF

(_, / (_(_( Rue de la touche lambert

35517 Cesson-Sevigne Cedex
FRANCE

Tel: +33 (0)2 99 87 68 91 Fax: +33 (0)2 99 84 11 30
E-mail: valette@…

Property mode set to 100644

File size: 111.4 KB

Line
1	/*
2	* arch/ppc/boot/em86real.S -- Small x86 emulator for video board setup
3	*
4	* Copyright (C) 1998 Gabriel Paubert, paubert@iram.es
5	*
6	* This file is subject to the terms and conditions of the GNU General Public
7	* License. See the file COPYING in the main directory of this archive
8	* for more details.
9	*/
10
11	/* If the symbol __BOOT__ is defined, a slightly different version is
12	* generated to be compiled with the -m relocatable option
13	*/
14
15	#ifdef __BOOT__
16	#include "bootldr.h"
17	/* It is impossible to gather statistics in the boot version */
18	#undef EIP_STATS
19	#endif
20
21	/*
22	*
23	* Given the size of this code, it deserves a few comments on how it works,
24	* and why it was implemented the way it is.
25	*
26	* The goal is to have a real mode i486SX emulator to initialize hardware,
27	* mostly graphics boards, by interpreting ROM BIOSes. The choice of a 486SX
28	* is logical since this is the lowest processor that PCI ROM BIOSes must run
29	* on.
30	*
31	* The goal of this emulator is not performance, but a small enough memory
32	* footprint to include it in a bootloader.
33	*
34	* It is actually likely to be comparable to a 25MHz 386DX on a 200MHz 603e !
35	* This is not as serious as it seems since most of the BIOS code performs
36	* a lot of accesses to I/O and non-cacheable memory spaces. For such
37	* instructions, the execution time is often dominated by bus accesses.
38	* Statistics of the code also shows that it spends a large function of
39	* the time in loops waiting for vertical retrace or programs one of the
40	* timers and waits for the count to go down to zero. This type of loop
41	* runs emulated at the same speed as on 5 GHz Pentium IV++ ;)
42	*
43	*/
44
45	/*
46	* Known bugs or differences with a real 486SX (real mode):
47	* - segment limits are not enforced (too costly)
48	* - xchg instructions with memory are not locked
49	* - lock prefixes are not implemented at all
50	* - long divides implemented but perhaps still buggy
51	* - miscellaneous system instructions not implemented
52	* (some probably cannot be implemented)
53	* - neither control nor debug registers are implemented for the time being
54	* (debug registers are impossible to implement at a reasonable cost)
55	*/
56
57	/* Code options, put them on the compiler command line */
58	/* #define EIP_STATS / / EIP based profiling */
59	/* #undef EIP_STATS */
60
61	/*
62	* Implementation notes:
63	*
64	* A) flags emulation.
65	*
66	* The most important decisions when it comes to obtain a reasonable speed
67	* are related to how the EFLAGS register is emulated.
68	*
69	* Note: the code to set up flags is complex, but it is only seldom
70	* executed since cmp and test instructions use much faster flag evaluation
71	* paths. For example the overflow flag is almost only needed for pushf and
72	* int. Comparison results only involve (SF^OF) or (SF^OF)+ZF and the
73	* implementation is fast in this case.
74	*
75	* Rarely used flags: AC, NT and IOPL are kept in a memory EFLAGS image.
76	* All other flags are either kept explicitly in PPC cr (DF, IF, and TF) or
77	* lazily evaluated from the state of 4 registers called flags, result, op1,
78	* op2, and sometimes the cr itself. The emulation has been designed for
79	* minimal overhead for the common case where the flags are never used. With
80	* few exceptions, all instructions that set flags leave the result of the
81	* computation in a register called result, and operands are taken from op1
82	* and op2 registers. However a few instructions like cmp, test and bit tests
83	* (bt/btc/btr/bts/bsf/bsr) explicitly set cr bits to short circuit
84	* condition code evaluation of conditional instructions.
85	*
86	* As a very brief summary:
87	*
88	* - the result of the last flag setting operation is often either in the
89	* result register or in op2 after increment or decrement instructions
90	* because result and op1 may be needed to compute the carry.
91	*
92	* - compare instruction leave the result of the unsigned comparison
93	* in cr4 and of signed comparison in cr6. This means that:
94	* - cr4[0]=CF (short circuit for jc/jnc)
95	* - cr4[1]=~(CF+ZF) (short circuit for ja/jna)
96	* - cr6[0]=(OF^SF) (short circuit for jl/jnl)
97	* - cr6[1]=~((SF^OF)+ZF) (short circuit for jg/jng)
98	* - cr6[2]=ZF (short circuit for jz/jnz)
99	*
100	* - test instruction set flags in cr6 and clear overflow. This means that:
101	* - cr6[0]=SF=(SF^OF) (short circuit for jl/jnl/js/jns)
102	* - cr6[1]=~((SF^OF)+ZF) (short circuit for jg/jng)
103	* - cr6[2]=ZF (short circuit for jz/jnz)
104	*
105	* All flags may be lazily evaluated from several values kept in registers:
106	*
107	* Flag: Depends upon:
108	* OF result, op1, op2, flags[INCDEC_FIELD,SUBTRACTING,OF_STATE_MASK]
109	* SF result, op2, flags[INCDEC_FIELD,RES_SIZE]
110	* ZF result, op2, cr6[2], flags[INCDEC_FIELD,RES_SIZE,ZF_PROTECT]
111	* AF op1, op2, flags[INCDEC_FIELD,SUBTRACTING,CF_IN]
112	* PF result, op2, flags[INCDEC_FIELD]
113	* CF result, op1, flags[CF_STATE_MASK, CF_IN]
114	*
115	* The order of the fields in the flags register has been chosen so that a
116	* single rlwimi is necessary for common instruction that do not affect all
117	* flags. (See the code for inc/dec emulation).
118	*
119	*
120	* B) opcodes and prefixes.
121	*
122	* The register called opcode holds in its low order 8 bits the opcode
123	* (second byte if the first byte is 0x0f). More precisely it holds the
124	* last byte fetched before the modrm byte or the immediate operand(s)
125	* of the instruction, if any. High order 24 bits are zero unless the
126	* instruction has prefixes. These higher order bits have the following
127	* meaning:
128	* 0x80000000 segment override prefix
129	* 0x00001000 repnz prefix (0xf2)
130	* 0x00000800 repz prefix (0xf3)
131	* 0x00000400 address size prefix (0x67)
132	* 0x00000200 operand size prefix (0x66)
133	* (bit 0x1000 and 0x800 cannot be set simultaneously)
134	*
135	* Therefore if there is a segment override the value will be between very
136	* negative (between 0x80000000 and 0x800016ff), if there is no segment
137	* override, the value will be between 0 and 0x16ff. The reason for
138	* this choice will be understood in the next part.
139	*
140	* C) addresing mode description tables.
141	*
142	* the encoding of the modrm bytes (especially in 16 bit mode) is quite
143	* complex. Hence a table, indexed by the five useful bits of the modrm
144	* byte is used to simplify decoding. Here is a description:
145	*
146	* bit mask meaning
147	* 0x80000000 use ss as default segment register
148	* 0x00004000 means that this addressing mode needs a base register
149	* (set for all entries except sib and displacement-only)
150	* 0x00002000 set if preceding is not set
151	* 0x00001000 set if an sib follows
152	* 0x00000700 base register to use (16 and 32 bit)
153	* 0x00000080 set in 32 bit addressing mode table, cleared in 16 bit
154	* (so extsb mask,entry; ori mask,mask,0xffff gives a mask)
155	* 0x00000070 kludge field, possible values are
156	* 0: 16 bit addressing mode without index
157	* 10: 32 bit addressing mode
158	* 60: 16 bit addressing mode with %si as index
159	* 70: 16 bit addressing mode with %di as index
160	*
161	* This convention leads to the following special values used to check for
162	* sib present and displacement-only, which happen to the three lowest
163	* values in the table (unsigned):
164	* 0x00003090 sib follows (implies it is a 32 bit mode)
165	* 0x00002090 32 bit displacement-only
166	* 0x00002000 16 bit displacement-only
167	*
168	* This means that all entries are either very negative in the 0x80002000
169	* range if the segment defaults to ss or higher than 0x2000 if it defaults
170	* to ds. Combined with the value in opcode this gives the following table:
171	* opcode entry entry>opcode ? segment to use
172	* positive positive yes ds (default)
173	* negative positive yes overriden by prefix
174	* positive negative no ss
175	* negative negative yes overridden by prefix
176	*
177	* Hence a simple comparison allows to check for the need to override
178	* the current base with ss, i.e., when ss is the default base and the
179	* instruction has no override prefix.
180	*
181	* D) BUGS
182	*
183	* This software is obviously bug-free :-). Nevertheless, if you encounter
184	* an interesting feature. Mail me a note, if possible with a detailed
185	* instruction example showing where and how it fails.
186	*
187	*/
188
189
190	/* Now the details of flag evaluation with the necessary macros */
191
192	/* Alignment check is toggable so the system believes it is a 486, but
193	CPUID is not to avoid unnecessary complexities. However, alignment
194	is actually never checked (real mode is CPL 0 anyway). */
195	#define AC86 13 /* Can only be toggled */
196	#define VM86 14 /* Not used for now */
197	#define RF86 15 /* Not emulated precisely */
198	/* Actually NT and IOPL are kept in memory */
199	#define NT86 17
200	#define IOPL86 18 /* Actually 18 and 19 */
201	#define OF86 20
202	#define DF86 21
203	#define IF86 22
204	#define TF86 23
205	#define SF86 24
206	#define ZF86 25
207	#define AF86 27
208	#define PF86 29
209	#define CF86 31
210
211	/* Where the less important flags are placed in PPC cr */
212	#define RF 20 /* Suppress trap flag: cr5[0] */
213	#define DF 21 /* Direction flag: cr5[1] */
214	#define IF 22 /* Interrupt flag: cr5[2] */
215	#define TF 23 /* Single step flag: cr5[3] */
216
217	/* Now the flags which are frequently used */
218	/*
219	* CF_IN is a copy of the input carry with PPC polarity,
220	* it is cleared for add, set for sub and cmp,
221	* equal to the x86 carry for adc and to its complement for sbb.
222	* it is used to evaluate AF and CF.
223	*/
224	#define CF_IN 0x80000000
225
226	/* #define GET_CF_IN(dst) rlwinm dst,flags,1,0x01 */
227
228	/* CF_IN_CR set in flags means that cr4[0] is a copy of carry bit */
229	#define CF_IN_CR 0x40000000
230
231	#define EVAL_CF andis. r3,flags,(CF_IN_CR)>>16; beql- _eval_cf
232
233	/*
234	* CF_STATE tells how to compute the carry bit.
235	* NOTRESULT16 and NOTRESULT8 are never set explicitly,
236	* but they may happen after a cmc instruction.
237	*/
238	#define CF 16 /* cr4[0] */
239	#define CF_LOCATION 0x30000000
240	#define CF_ZERO 0x00000000
241	#define CF_EXPLICIT 0x00000000
242	#define CF_COMPLEMENT 0x08000000 /* Indeed a polarity bit */
243	#define CF_STATE_MASK (CF_LOCATION\|CF_COMPLEMENT)
244	#define CF_VALUE 0x08000000
245	#define CF_SET 0x08000000
246	#define CF_RES32 0x10000000
247	#define CF_NOTRES32 0x18000000
248	#define CF_RES16 0x20000000
249	#define CF_NOTRES16 0x28000000
250	#define CF_RES8 0x30000000
251	#define CF_NOTRES8 0x38000000
252
253	#define CF_ADDL CF_RES32
254	#define CF_SUBL CF_NOTRES32
255	#define CF_ADDW CF_RES16
256	#define CF_SUBW CF_RES16
257	#define CF_ADDB CF_RES8
258	#define CF_SUBB CF_RES8
259
260	#define CF_ROTCNT(dst) rlwinm dst,flags,7,0x18
261	#define CF_POL(dst,pos) rlwinm dst,flags,(36-pos)%32,pos,pos
262	#define CF_POL_INSERT(dst,pos) \
263	rlwimi dst,flags,(36-pos)%32,pos,pos
264	#define RES2CF(dst) rlwinm dst,result,8,7,15
265
266	/*
267	* OF_STATE tells how to compute the overflow bit. When the low order bit
268	* is set (OF_EXPLICIT), it means that OF is the exclusive or of the
269	* two other bits. For the reason of this choice, see rotate instructions.
270	*/
271	#define OF 1 /* Only after EVAL_OF */
272	#define OF_STATE_MASK 0x07000000
273	#define OF_INCDEC 0x00000000
274	#define OF_EXPLICIT 0x01000000
275	#define OF_ZERO 0x01000000
276	#define OF_VALUE 0x04000000
277	#define OF_SET 0x04000000
278	#define OF_ONE 0x05000000
279	#define OF_XOR 0x06000000
280	#define OF_ARITHL 0x06000000
281	#define OF_ARITHW 0x02000000
282	#define OF_ARITHB 0x04000000
283
284	#define EVAL_OF rlwinm. r3,flags,6,0,1; bngl+ _eval_of; andis. r3,flags,OF_VALUE>>16
285
286	/* See _eval_of to see how this can be used */
287	#define OF_ROTCNT(dst) rlwinm dst,flags,10,0x1c
288
289	/*
290	* SIGNED_IN_CR means that cr6 is set as after a signed compare:
291	* - cr6[0] is SF^OF for jl/jnl/setl/setnl...
292	* - cr6[1] is ~((SF^OF)+ZF) for jg/jng/setg/setng...
293	* - cr6[2] is ZF (ZF_IN_CR is always set if this bit is set)
294	*/
295	#define SLT 24 /* cr6[0], signed less than */
296	#define SGT 25 /* cr6[1], signed greater than */
297	#define SIGNED_IN_CR 0x00800000
298
299	#define EVAL_SIGNED andis. r3,flags,SIGNED_IN_CR>>16; beql- _eval_signed
300
301	/*
302	* Above in CR means that cr4 is set as after an unsigned compare:
303	* - cr4[0] is CF (CF_IN_CR is also set)
304	* - cr4[1] is ~(CF+ZF) (ZF_IN_CR is also set)
305	*/
306	#define ABOVE 17 /* cr4[1] */
307	#define ABOVE_IN_CR 0x00400000
308
309	#define EVAL_ABOVE andis. r3,flags,ABOVE_IN_CR>>16; beql- _eval_above
310
311	/* SF_IN_CR means cr6[0] is a copy of SF. It implies ZF_IN_CR is also set */
312	#define SF 24 /* cr6[0] */
313	#define SF_IN_CR 0x00200000
314
315	#define EVAL_SF andis. r3,flags,SF_IN_CR>>16; beql- _eval_sf_zf
316
317	/* ZF_IN_CR means cr6[2] is a copy of ZF. */
318	#define ZF 26
319	#define ZF_IN_CR 0x00100000
320
321	#define EVAL_ZF andis. r3,flags,ZF_IN_CR>>16; beql- _eval_sf_zf
322	#define ZF2ZF86(s,d) rlwimi d,s,ZF-ZF86,ZF86,ZF86
323	#define ZF862ZF(reg) rlwimi reg,reg,32+ZF86-ZF,ZF,ZF
324
325	/*
326	* ZF_PROTECT means cr6[2] is the only valid value for ZF. This is necessary
327	* because some infrequent instructions may leave SF and ZF in an apparently
328	* inconsistent state (both set): sahf, popf and the few (not implemented)
329	* instructions that only affect ZF.
330	*/
331	#define ZF_PROTECT 0x00080000
332
333	/* The parity is always evaluated when it is needed */
334	#define PF 0 /* Only after EVAL_PF */
335	#define EVAL_PF bl _eval_pf
336
337	/* This field gives the shift amount to use to evaluate SF
338	and ZF when ZF_PROTECT is not set */
339	#define RES_SIZE_MASK 0x00060000
340	#define RESL 0x00000000
341	#define RESW 0x00040000
342	#define RESB 0x00060000
343
344	#define RES_SHIFT(dst) rlwinm dst,flags,18,0x18
345
346	/* SUBTRACTING is set if the last flag setting instruction was sub/sbb/cmp,
347	used to evaluate OF and AF */
348	#define SUBTRACTING 0x00010000
349
350	#define GET_ADDSUB(dst) rlwinm dst,flags,16,0x01
351
352	/* rotate (rcl/rcr/rol/ror) affect CF and OF but not other flags */
353	#define ROTATE_MASK (CF_IN_CR\|CF_STATE_MASK\|ABOVE_IN_CR\|OF_STATE_MASK\|SIGNED_IN_CR)
354	#define ROTATE_FLAGS rlwimi flags,one,24,ROTATE_MASK
355
356	/*
357	* INCDEC_FIELD has at most one bit set when the last flag setting instruction
358	* was either inc or dec (which do not affect the carry). When one of these
359	* bits is set, it affects the way OF, SF, ZF, AF, and PF are evaluated.
360	*/
361	#define INCDEC_FIELD 0x0000ff00
362
363	#define DECB_SHIFT 8
364	#define INCB_SHIFT 9
365	#define DECW_SHIFT 10
366	#define INCW_SHIFT 11
367	#define DECL_SHIFT 14
368	#define INCL_SHIFT 15
369
370	#define INCDEC_MASK (OF_STATE_MASK\|SIGNED_IN_CR\|ABOVE_IN_CR\|SF_IN_CR\|\
371	ZF_IN_CR\|ZF_PROTECT\|RES_SIZE_MASK\|SUBTRACTING\|\
372	INCDEC_FIELD)
373	/* Operations to perform to tell where the flags are after inc or dec */
374	#define INC_FLAGS(BWL) rlwimi flags,one,INC##BWL##_SHIFT,INCDEC_MASK
375	#define DEC_FLAGS(BWL) rlwimi flags,one,DEC##BWL##_SHIFT,INCDEC_MASK
376
377	/* How the flags are set after arithmetic operations */
378	#define FLAGS_ADD(BWL) (CF_ADD##BWL\|OF_ARITH##BWL\|RES##BWL)
379	#define FLAGS_SBB(BWL) (CF_SUB##BWL\|OF_ARITH##BWL\|RES##BWL\|SUBTRACTING)
380	#define FLAGS_SUB(BWL) FLAGS_SBB(BWL)\|CF_IN
381	#define FLAGS_CMP(BWL) FLAGS_SUB(BWL)\|ZF_IN_CR\|CF_IN_CR\|SIGNED_IN_CR\|ABOVE_IN_CR
382
383	/* How the flags are set after logical operations */
384	#define FLAGS_LOG(BWL) (CF_ZERO\|OF_ZERO\|RES##BWL)
385	#define FLAGS_TEST(BWL) FLAGS_LOG(BWL)\|ZF_IN_CR\|SIGNED_IN_CR\|SF_IN_CR
386
387	/* How the flags are set after bt/btc/btr/bts. */
388	#define FLAGS_BTEST CF_IN_CR\|CF_ADDL\|OF_ZERO\|RESL
389
390	/* How the flags are set after bsf/bsr. */
391	#define FLAGS_BSRCH(WL) CF_ZERO\|OF_ZERO\|RES##WL\|ZF_IN_CR
392
393	/* How the flags are set after logical right shifts */
394	#define FLAGS_SHR(BWL) (CF_EXPLICIT\|OF_ARITH##BWL\|RES##BWL)
395
396	/* How the flags are set after double length shifts */
397	#define FLAGS_DBLSH(WL) (CF_EXPLICIT\|OF_ARITH##WL\|RES##WL)
398
399	/* How the flags are set after multiplies */
400	#define FLAGS_MUL (CF_EXPLICIT\|OF_EXPLICIT)
401
402	#define SET_FLAGS(fl) lis flags,(fl)>>16
403	#define ADD_FLAGS(fl) addis flags,flags,(fl)>>16
404
405	/*
406	* We are always off by one when compared with Intel's eip, this shortens
407	* code by allowing to load next byte with lbzu x,1(eip). The register
408	* called eip actually contains csbase+eip, and thus should be called lip
409	* for linear ip.
410	*/
411
412	/*
413	* Reason codes passed to the C part of the emulator, this includes all
414	* instructions which may change the current code segment. These definitions
415	* will soon go into a separate include file. Codes 0 to 255 correspond
416	* directly to the interrupt/trap that has to be generated.
417	*/
418
419	#define code_divide_err 0
420	#define code_trap 1
421	#define code_int3 3
422	#define code_into 4
423	#define code_bound 5
424	#define code_ud 6
425	#define code_dna 7 /* FPU not available */
426
427	#define code_iretw 256 /* Interrupt returns */
428	#define code_iretl 257
429	#define code_lcallw 258 /* Far calls and jumps */
430	#define code_lcalll 259
431	#define code_ljmpw 260
432	#define code_ljmpl 261
433	#define code_lretw 262 /* Far returns */
434	#define code_lretl 263
435	#define code_softint 264 /* int $xx */
436	#define code_lock 265 /* Lock prefix */
437	/* Codes 1024 to 2047 are used for I/O port access instructions:
438	- The three LSB define the port size (1, 2 or 4)
439	- bit of weight 512 means out if set, in if clear
440	- bit of weight 256 means ins/outs if set, in/out if clear
441	- bit of weight 128 means use 32 bit addresses if set, 16 bit if clear
442	(only used for ins/outs instructions, always clear for in/out)
443	*/
444	#define code_inb 1024+1
445	#define code_inw 1024+2
446	#define code_inl 1024+4
447	#define code_outb 1024+512+1
448	#define code_outw 1024+512+2
449	#define code_outl 1024+512+4
450	#define code_insb_a16 1024+256+1
451	#define code_insw_a16 1024+256+2
452	#define code_insl_a16 1024+256+4
453	#define code_outsb_a16 1024+512+256+1
454	#define code_outsw_a16 1024+512+256+2
455	#define code_outsl_a16 1024+512+256+4
456	#define code_insb_a32 1024+256+128+1
457	#define code_insw_a32 1024+256+128+2
458	#define code_insl_a32 1024+256+128+4
459	#define code_outsb_a32 1024+512+256+128+1
460	#define code_outsw_a32 1024+512+256+128+2
461	#define code_outsl_a32 1024+512+256+128+4
462
463	#define state 31
464	/* r31 (state) is a pointer to a structure describing the emulated x86
465	processor, its layout is the following:
466
467	first the general purpose registers, they are in little endian byte order
468
469	offset name
470
471	0 eax/ax/al
472	1 ah
473	4 ecx/cx/cl
474	5 ch
475	8 edx/dx/dl
476	9 dh
477	12 ebx/bx/bl
478	13 bh
479	16 esp/sp
480	20 ebp/bp
481	24 esi/si
482	28 edi/di
483	*/
484
485	#define AL 0
486	#define AX 0
487	#define EAX 0
488	#define AH 1
489	#define CL 4
490	#define CX 4
491	#define ECX 4
492	#define DX 8
493	#define EDX 8
494	#define BX 12
495	#define EBX 12
496	#define SP 16
497	#define ESP 16
498	#define BP 20
499	#define EBP 20
500	#define SI 24
501	#define ESI 24
502	#define DI 28
503	#define EDI 28
504
505	/*
506	than the rest of the machine state, big endian !
507
508	offset name
509
510	32 essel segment register selectors (values)
511	36 cssel
512	40 sssel
513	44 dssel
514	48 fssel
515	52 gssel
516	56 eipimg true eip (register named eip is csbase+eip)
517	60 eflags eip and eflags only valid when C code running !
518	64 esbase segment registers bases
519	68 csbase
520	72 ssbase
521	76 dsbase
522	80 fsbase
523	84 gsbase
524	88 iobase For I/O instructions, I/O space virtual base
525	92 ioperm I/O permission bitmap pointer
526	96 reason Reason code when calling external emulator
527	100 nexteip eip past instruction for external emulator
528	104 parm1 parameter for external emulator
529	108 parm2 parameter for external emulator
530	112 _opcode current opcode register for external emulator
531	116 _base segment register base for external emulator
532	120 _offset intruction operand offset
533	More internal state was dumped here for debugging in first versions
534
535	128 vbase where the 1Mb memory is mapped
536	132 cntimg instruction counter
537	136 scratch
538	192 eipstat array of 32k unsigned long pairs for eip stats
539	*/
540
541	#define essel 32
542	#define cssel 36
543	#define sssel 40
544	#define dssel 44
545	#define fssel 48
546	#define gssel 52
547	#define eipimg 56
548	#define eflags 60
549	#define esbase 64
550	#define csbase 68
551	#define ssbase 72
552	#define dsbase 76
553	#define fsbase 80
554	#define gsbase 84
555	#define iobase 88
556	#define ioperm 92
557	#define reason 96
558	#define nexteip 100
559	#define parm1 104
560	#define parm2 108
561	#define _opcode 112
562	#define _base 116
563	#define _offset 120
564	#define vbase 128
565	#define cntimg 132
566	#ifdef EIP_STATS
567	#define eipstat 192
568	#endif
569	/* Global registers */
570
571	/* Some segment register bases are permanently kept in registers since they
572	are often used: these are csb, esb and ssb because they are
573	required for jumps, string instructions, and pushes/pops/calls/rets.
574	dsbase is not kept in a register but loaded from memory to allow somewhat
575	more parallelism in the main emulation loop.
576	*/
577
578	#define one 30 /* Constant one, so pervasive */
579	#define ssb 29
580	#define csb 28
581	#define esb 27
582	#define eip 26 /* That one is indeed csbase+(e)ip-1 */
583	#define result 25 /* For the use of result, op1, op2 */
584	#define op1 24 /* see the section on flag emulation */
585	#define op2 23
586	#define opbase 22 /* default opcode table */
587	#define flags 21 /* See earlier description */
588	#define opcode 20 /* Opcode */
589	#define opreg 19 /* Opcode extension/register number */
590	/* base is reloaded with the base of the ds segment at the beginning of
591	every instruction, it is modified by segment override prefixes, when
592	the default base segment is ss, or when the modrm byte specifies a
593	register operand */
594	#define base 18 /* Instruction's operand segment base */
595	#define offset 17 /* Instruction's memory operand offset */
596	/* used to address a table telling how to decode the addressing mode
597	specified by the modrm byte */
598	#define adbase 16 /* addressing mode table */
599	/* Following registers are used only as dedicated temporaries during decoding,
600	they are free for use during emulation */
601	/*
602	* ceip (current eip) is only in use when we call the external emulator for
603	* instructions that fault. Note that it is forbidden to change flags before
604	* the check for the fault happens (divide by zero...) ! ceip is also used
605	* when measuring timing.
606	*/
607	#define ceip 15
608
609	/* A register used to measure timing information (when enabled) */
610	#ifdef EIP_STATS
611	#define tstamp 14
612	#endif
613
614	#define count 12 /* Instruction counter. */
615
616	#define r0 0
617	#define r1 1 /* PPC Stack pointer. */
618	#define r3 3
619	#define r4 4
620	#define r5 5
621	#define r6 6
622	#define r7 7
623
624	/* Macros to read code stream */
625	#define NEXTBYTE(dest) lbzu dest,1(eip)
626	#define NEXTWORD(dest) lhbrx dest,eip,one; la eip,2(eip)
627	#define NEXTDWORD(dest) lwbrx dest,eip,one; la eip,4(eip)
628	#define NEXT b nop
629	#define GOTNEXT b gotopcode
630
631	#ifdef __BOOT__
632	START_GOT
633	GOT_ENTRY(_jtables)
634	GOT_ENTRY(jtab_www)
635	GOT_ENTRY(adtable)
636	END_GOT
637	#else
638	.text
639	#endif
640	.align 2
641	.global em86_enter
642	.type em86_enter,@function
643	em86_enter: stwu r1,-96(r1) # allocate stack
644	mflr r0
645	stmw 14,24(r1)
646	mfcr r4
647	stw r0,100(r1)
648	mr state,r3
649	stw r4,20(r1)
650	#ifdef __BOOT__
651	/* We need this since r30 is the default GOT pointer */
652	#define r30 30
653	GET_GOT
654	/* The relocation of these tables is explicit, this could be done
655	* automatically with fixups but would add more than 8kb in the fixup tables.
656	*/
657	lwz r3,GOT(_jtables)
658	lwz r4,_endjtables-_jtables(r3)
659	sub. r4,r3,r4
660	beq+ 1f
661	li r0,((_endjtables-_jtables)>>2)+1
662	addi r3,r3,-4
663	mtctr r0
664	0: lwzu r5,4(r3)
665	add r5,r5,r4
666	stw r5,0(r3)
667	bdnz 0b
668	1: lwz adbase,GOT(adtable)
669	lwz opbase,GOT(jtab_www)
670	/* Now r30 is only used as constant 1 */
671	#undef r30
672	li one,1 # pervasive constant
673	#else
674	lis opbase,jtab_www@ha
675	lis adbase,adtable@ha
676	li one,1 # pervasive constant
677	addi opbase,opbase,jtab_www@l
678	addi adbase,adbase,adtable@l
679	#ifdef EIP_STATS
680	li ceip,0
681	mftb tstamp
682	#endif
683	#endif
684	/* We branch back here when calling an external function tells us to resume */
685	restart: lwz r3,eflags(state)
686	lis flags,(OF_EXPLICIT\|ZF_IN_CR\|ZF_PROTECT\|SF_IN_CR)>>16
687	lwz csb,csbase(state)
688	extsb result,r3 # SF/PF
689	rlwinm op1,r3,31,0x08 # AF
690	lwz eip,eipimg(state)
691	ZF862ZF(r3) # cr6
692	addi op2,op1,0 # AF
693	lwz ssb,ssbase(state)
694	rlwimi flags,r3,15,OF_VALUE # OF
695	rlwimi r3,r3,32+RF86-RF,RF,RF # RF
696	lwz esb,esbase(state)
697	ori result,result,0xfb # PF
698	mtcrf 0x06,r3 # RF/DF/IF/TF/SF/ZF
699	lbzux opcode,eip,csb
700	rlwimi flags,r3,27,CF_VALUE # CF
701	xori result,result,0xff # PF
702	lwz count,cntimg(state)
703	GOTNEXT # start the emulator
704
705	/* Now return */
706	exit: lwz r0,100(r1)
707	lwz r4,20(r1)
708	mtlr r0
709	lmw 14,24(r1)
710	mtcr r4
711	addi r1,r1,96
712	blr
713
714	trap: crmove 0,RF
715	crclr RF
716	bt- 0,resume
717	sub ceip,eip,csb
718	li r3,code_trap
719	complex: addi eip,eip,1
720	stw r3,reason(state)
721	sub eip,eip,csb
722	stw op1,240(state)
723	stw op2,244(state)
724	stw result,248(state)
725	stw flags,252(state)
726	stw r4,parm1(state)
727	stw r5,parm2(state)
728	stw opcode,_opcode(state)
729	bl _eval_flags
730	stw base,_base(state)
731	stw eip,nexteip(state)
732	stw r3,eflags(state)
733	mr r3,state
734	stw offset,_offset(state)
735	stw ceip,eipimg(state)
736	stw count,cntimg(state)
737	bl em86_trap
738	cmpwi r3,0
739	bne exit
740	b restart
741
742	/* Main loop */
743	/*
744	* The two LSB of each entry in the main table mean the following:
745	* 00: indirect opcode: modrm follows and the three middle bits are an
746	* opcode extension. The entry points to another jump table.
747	* 01: direct instruction, branch directly to the routine.
748	* 10: modrm specifies byte size memory and register operands.
749	* 11: modrm specifies word/long memory and register operands.
750	*
751	* The modrm byte, if present, is always loaded in r7.
752	*
753	* Note: most "mr x,y" instructions have been replaced by "addi x,y,0" since
754	* the latter can be executed in the second integer unit on 603e.
755	*/
756
757	/*
758	* This code is very good example of absolutely unmaintainable code.
759	* It was actually much easier to write than it is to understand !
760	* If my computations are right, the maximum path length from fetching
761	* the opcode to exiting to the actual instruction execution is
762	* 46 instructions (for non-prefixed, single byte opcode instructions).
763	*
764	*/
765	.align 5
766	#ifdef EIP_STATS
767	nop: NEXTBYTE(opcode)
768	gotopcode: slwi r3,opcode,2
769	bt- TF,trap
770	resume: lwzx r4,opbase,r3
771	addi r5,state,eipstat+4
772	clrlslwi r6,ceip,17,3
773	mtctr r4
774	lwzux r7,r5,r6
775	slwi. r0,r4,30 # two lsb of table entry
776	sub r7,r7,tstamp
777	lwz r6,-4(r5)
778	mftb tstamp
779	addi r6,r6,1
780	sub ceip,eip,csb
781	stw r6,-4(r5)
782	add r7,r7,tstamp
783	lwz base,dsbase(state)
784	stw r7,0(r5)
785	#else
786	nop: NEXTBYTE(opcode)
787	gotopcode: slwi r3,opcode,2
788	bt- TF,trap
789	resume: lwzx r4,opbase,r3
790	sub ceip,eip,csb
791	mtctr r4
792	slwi. r0,r4,30 # two lsb of table entry
793	lwz base,dsbase(state)
794	addi count,count,1
795	#endif
796	bgtctr- # for instructions without modrm
797
798	/* modrm byte present */
799	NEXTBYTE(r7) # modrm byte
800	cmplwi cr1,r7,192
801	rlwinm opreg,r7,31,0x1c
802	beq- cr0,8f # extended opcode
803	/* modrm with middle 3 bits specifying a register (non prefixed) */
804	rlwinm r0,r4,3,0x8
805	li r4,0x1c0d
806	rlwimi opreg,r7,27,0x01
807	srw r4,r4,r0
808	and opreg,opreg,r4
809	blt cr1,9f
810	/* modrm with 2 register operands */
811	1: rlwinm offset,r7,2,0x1c
812	addi base,state,0
813	rlwimi offset,r7,30,0x01
814	and offset,offset,r4
815	bctr
816
817	/* Prefixes: first segment overrides */
818	.align 4
819	_es: NEXTBYTE(r7); addi base,esb,0
820	oris opcode,opcode,0x8000; b 2f
821	_cs: NEXTBYTE(r7); addi base,csb,0
822	oris opcode,opcode,0x8000; b 2f
823	_fs: NEXTBYTE(r7); lwz base,fsbase(state)
824	oris opcode,opcode,0x8000; b 2f
825	_gs: NEXTBYTE(r7); lwz base,gsbase(state)
826	oris opcode,opcode,0x8000; b 2f
827	_ss: NEXTBYTE(r7); addi base,ssb,0
828	oris opcode,opcode,0x8000; b 2f
829	_ds: NEXTBYTE(r7)
830	oris opcode,opcode,0x8000; b 2f
831
832	/* Lock (unimplemented) and repeat prefixes */
833	_lock: li r3,code_lock; b complex
834	_repnz: NEXTBYTE(r7); rlwimi opcode,one,12,0x1800; b 2f
835	_repz: NEXTBYTE(r7); rlwimi opcode,one,11,0x1800; b 2f
836
837	/* Operand and address size prefixes */
838	.align 4
839	_opsize: NEXTBYTE(r7); ori opcode,opcode,0x200
840	rlwinm r3,opcode,2,0x1ffc; b 2f
841	_adsize: NEXTBYTE(r7); ori opcode,opcode,0x400
842	rlwinm r3,opcode,2,0x1ffc; b 2f
843
844	_twobytes: NEXTBYTE(r7); addi r3,r3,0x400
845	2: rlwimi r3,r7,2,0x3fc
846	lwzx r4,opbase,r3
847	rlwimi opcode,r7,0,0xff
848	mtctr r4
849	slwi. r0,r4,30
850	bgtctr- # direct instruction
851	/* modrm byte in a prefixed instruction */
852	NEXTBYTE(r7) # modrm byte
853	cmpwi cr1,r7,192
854	rlwinm opreg,r7,31,0x1c
855	beq- 6f
856	/* modrm with middle 3 bits specifying a register (prefixed) */
857	rlwinm r0,r4,3,0x8
858	li r4,0x1c0d
859	rlwimi opreg,r7,27,0x01
860	srw r4,r4,r0
861	and opreg,opreg,r4
862	bnl cr1,1b # 2 register operands
863	/* modrm specifying memory with prefix */
864	3: rlwinm r3,r3,27,0xff80
865	rlwimi adbase,r7,2,0x1c
866	extsh r3,r3
867	rlwimi r3,r7,31,0x60
868	lwzx r4,r3,adbase
869	cmpwi cr1,r4,0x3090
870	bnl+ cr1,10f
871	/* displacement only addressing modes */
872	4: cmpwi r4,0x2000
873	bne 5f
874	NEXTWORD(offset)
875	bctr
876	5: NEXTDWORD(offset)
877	bctr
878	/* modrm with opcode extension (prefixed) */
879	6: lwzx r4,r4,opreg
880	mtctr r4
881	blt cr1,3b
882	/* modrm with opcode extension and register operand */
883	7: rlwinm offset,r7,2,0x1c
884	addi base,state,0
885	rlwinm r0,r4,3,0x8
886	li r4,0x1c0d
887	rlwimi offset,r7,30,0x01
888	srw r4,r4,r0
889	and offset,offset,r4
890	bctr
891	/* modrm with opcode extension (non prefixed) */
892	8: lwzx r4,r4,opreg
893	mtctr r4
894	/* FIXME ? We continue fetching even if the opcode extension is undefined.
895	* It shouldn't do any harm on real mode emulation anyway, and for ROM
896	* BIOS emulation, we are supposed to read valid code.
897	*/
898	bnl cr1,7b
899	/* modrm specifying memory without prefix */
900	9: rlwimi adbase,r7,2,0x1c # memory addressing mode computation
901	rlwinm r3,r7,31,0x60
902	lwzx r4,r3,adbase
903	cmplwi cr1,r4,0x3090
904	blt- cr1,4b # displacement only addressing mode
905	10: rlwinm. r0,r7,24,0,1 # three cases distinguished
906	beq- cr1,15f # an sib follows
907	rlwinm r3,r4,30,0x1c # 16bit/32bit/%si index/%di index
908	cmpwi cr1,r3,8 # set cr1 as early as possible
909	rlwinm r6,r4,26,0x1c # base register
910	lwbrx offset,state,r6 # load the base register
911	beq cr0,14f # no displacement
912	cmpw cr2,r4,opcode # check for ss as default base
913	bgt cr0,12f # byte offset
914	beq cr1,11f # 32 bit displacement
915	NEXTWORD(r5) # 16 bit displacement
916	bgt cr1,13f # d16(base,index)
917	/* d16(base) */
918	add offset,offset,r5
919	clrlwi offset,offset,16
920	bgtctr cr2
921	addi base,ssb,0
922	bctr
923	/* d32(base) */
924	11: NEXTDWORD(r5)
925	add offset,offset,r5
926	bgtctr cr2
927	addi base,ssb,0
928	bctr
929	/* 8 bit displacement */
930	12: NEXTBYTE(r5)
931	extsb r5,r5
932	bgt cr1,13f
933	/* d8(base) */
934	extsb r6,r4
935	add offset,offset,r5
936	ori r6,r6,0xffff
937	and offset,offset,r6
938	bgtctr cr2
939	addi base,ssb,0
940	bctr
941	/* d8(base,index) and d16(base,index) share this code ! */
942	13: lhbrx r3,state,r3
943	add offset,offset,r5
944	add offset,offset,r3
945	clrlwi offset,offset,16
946	bgtctr cr2
947	addi base,ssb,0
948	bctr
949	/* no displacement: only indexed modes may use ss as default base */
950	14: beqctr cr1 # 32 bit register indirect
951	clrlwi offset,offset,16
952	bltctr cr1 # 16 bit register indirect
953	/* (base,index) */
954	lhbrx r3,state,r3 # 16 bit [{bp,bx}+{si,di}]
955	cmpw cr2,r4,opcode # check for ss as default base
956	add offset,offset,r3
957	clrlwi offset,offset,r3
958	bgtctr+ cr2
959	addi base,ssb,0
960	bctr
961	/* sib modes, note that the size of the offset can be known from cr0 */
962	15: NEXTBYTE(r7) # get sib
963	rlwinm r3,r7,31,0x1c # index
964	rlwinm offset,r7,2,0x1c # base
965	cmpwi cr1,r3,ESP # has index ?
966	bne cr0,18f # base+d8/d32
967	cmpwi offset,EBP
968	beq 17f # d32(,index,scale)
969	xori r4,one,0xcc01 # build 0x0000cc00
970	rlwnm r4,r4,offset,0,1 # 0 or 0xc0000000
971	lwbrx offset,state,offset
972	cmpw cr2,r4,opcode # use ss ?
973	beq- cr1,16f # no index
974	/* (base,index,scale) */
975	lwbrx r3,state,r3
976	srwi r6,r7,6
977	slw r3,r3,r6
978	add offset,offset,r3
979	bgtctr cr2
980	addi base,ssb,0
981	bctr
982	/* (base), in practice only (%esp) is coded this way */
983	16: bgtctr cr2
984	addi base,ssb,0
985	bctr
986	/* d32(,index,scale) */
987	17: NEXTDWORD(offset)
988	beqctr- cr1 # no index: very unlikely
989	lwbrx r3,state,r3
990	srwi r6,r7,6
991	slw r3,r3,r6
992	add offset,offset,r3
993	bctr
994	/* 8 or 32 bit displacement */
995	18: xori r4,one,0xcc01 # build 0x0000cc00
996	rlwnm r4,r4,offset,0,1 # 0 or 0xc0000000
997	lwbrx offset,state,offset
998	cmpw cr2,r4,opcode # use ss ?
999	bgt cr0,20f # 8 bit offset
1000	/* 32 bit displacement */
1001	NEXTDWORD(r5)
1002	beq- cr1,21f
1003	/* d(base,index,scale) */
1004	19: lwbrx r3,state,r3
1005	add offset,offset,r5
1006	add offset,offset,r3
1007	bgtctr cr2
1008	addi base,ssb,0
1009	bctr
1010	/* 8 bit displacement */
1011	20: NEXTBYTE(r5)
1012	extsb r5,r5
1013	bne+ cr1,19b
1014	/* d(base), in practice base is %esp */
1015	21: add offset,offset,r5
1016	bgtctr- cr2
1017	addi base,ssb,0
1018	bctr
1019
1020	/*
1021	* Flag evaluation subroutines: they have not been written for performance
1022	* since they are not often used in practice. The rule of the game was to
1023	* write them with as few branches as possible.
1024	* The first routines eveluate either one or 2 (ZF and SF simultaneously)
1025	* flags and do not use r0 and r7.
1026	* The more complex routines (_eval_above, _eval_signed and _eval_flags)
1027	* call the former ones, using r0 as a return address save register and
1028	* r7 as a safe temporary.
1029	*/
1030
1031	/*
1032	* _eval_sf_zf evaluates simultaneously SF and ZF unless ZF is already valid
1033	* and protected because it is possible, although it is exceptional, to have
1034	* SF and ZF set at the same time after a few instructions which may leave the
1035	* flags in this apparently inconsistent state: sahf, popf, iret and the few
1036	* (for now unimplemented) instructions which only affect ZF (lar, lsl, arpl,
1037	* cmpxchg8b). This also solves the obscure case of ZF set and PF clear.
1038	* On return: SF=cr6[0], ZF=cr6[2].
1039	*/
1040
1041	_eval_sf_zf: andis. r5,flags,ZF_PROTECT>>16
1042	rlwinm r3,flags,0,INCDEC_FIELD
1043	RES_SHIFT(r4)
1044	cntlzw r3,r3
1045	slw r4,result,r4
1046	srwi r5,r3,5 # ? use result : use op1
1047	rlwinm r3,r3,2,0x18
1048	oris flags,flags,(SF_IN_CR\|SIGNED_IN_CR\|ZF_IN_CR)>>16
1049	neg r5,r5 # mux result/op2
1050	slw r3,op2,r3
1051	and r4,r4,r5
1052	andc r3,r3,r5
1053	xoris flags,flags,(SIGNED_IN_CR)>>16
1054	bne- 1f # 12 instructions between set
1055	or r3,r3,r4 # and test, good for folding
1056	cmpwi cr6,r3,0
1057	blr
1058	1: or. r3,r3,r4
1059	crmove SF,0
1060	blr
1061
1062	/*
1063	* _eval_cf may be called at any time, no other flag is affected.
1064	* On return: CF=cr4[0], r3= CF ? 0x100:0 = CF<<8.
1065	*/
1066	_eval_cf: addc r3,flags,flags # CF_IN to xer[ca]
1067	RES2CF(r4) # get 8 or 16 bit carry
1068	subfe r3,result,op1 # generate PPC carry for
1069	CF_ROTCNT(r5) # preceding operation
1070	addze r3,r4 # put carry into LSB
1071	CF_POL(r4,23) # polarity & 0x100
1072	oris flags,flags,(CF_IN_CR\|ABOVE_IN_CR)>>16
1073	rlwnm r3,r3,r5,23,23 # shift carry there
1074	xor r3,r3,r4 # CF <<8
1075	xoris flags,flags,(ABOVE_IN_CR)>>16
1076	cmplw cr4,one,r3 # sets cr4[0]
1077	blr
1078
1079	/*
1080	* eval_of returns the overflow flag in OF_STATE field, which will be
1081	* either 001 (OF clear) or 101 (OF set), is is only called when the two
1082	* low order bits of OF_STATE are not 01 (otherwise it will work but
1083	* it is an elaborate variant of a nop with a few registers destroyed)
1084	* The code multiplexes several sources in a branchless way, was fun to write.
1085	*/
1086	_eval_of: GET_ADDSUB(r4) # 0(add)/1(sub)
1087	rlwinm r3,flags,0,INCDEC_FIELD
1088	neg r4,r4 # 0(add)/-1(sub)
1089	eqv r5,result,op1 # result[]==op1[] (bit by bit)
1090	cntlzw r3,r3 # inc/dec
1091	xor r4,r4,op2 # true sign of op2
1092	oris r5,r5,0x0808 # bits to clear
1093	clrlwi r6,r3,31 # 0(inc)/1(dec)
1094	eqv r4,r4,op1 # op1[]==op2[] (bit by bit)
1095	add r6,op2,r6 # add 1 if dec
1096	rlwinm r3,r3,2,0x18 # incdec_shift
1097	andc r4,r4,r5 # arithmetic overflow
1098	slw r3,r6,r3 # shifted inc/dec result
1099	addis r3,r3,0x8000 # compare with 0x80000000
1100	ori r4,r4,0x0808 # bits to set
1101	cntlzw r3,r3 # 32 if inc/dec overflow
1102	OF_ROTCNT(r6)
1103	rlwimi r4,r3,18,0x00800000 # insert inc/dec overflow
1104	rlwimi flags,one,24,OF_STATE_MASK
1105	rlwnm r3,r4,r6,8,8 # get field
1106	rlwimi flags,r3,3,OF_VALUE # insert OF
1107	blr
1108
1109	/*
1110	* _eval_pf will always be called when needed (complex but infrequent),
1111	* there are a few quirks for a branchless solution.
1112	* On return: PF=cr0[0], PF=MSB(r3)
1113	*/
1114	_eval_pf: rlwinm r3,flags,0,INCDEC_FIELD
1115	rotrwi r4,op2,4 # from inc/dec
1116	rotrwi r5,result,4 # from result
1117	cntlzw r3,r3 # use result if 32
1118	xor r4,r4,op2
1119	xor r5,r5,result
1120	rlwinm r3,r3,26,0,0 # 32 becomes 0x80000000
1121	clrlwi r4,r4,28
1122	lis r6,0x9669 # constant to shift
1123	clrlwi r5,r5,28
1124	rlwnm r4,r6,r4,0,0 # parity from inc/dec
1125	rlwnm r5,r6,r5,0,0 # parity from result
1126	andc r4,r4,r3 # select which one
1127	and r5,r5,r3
1128	add. r3,r4,r5 # and test to simplify
1129	blr # returns in r3 and cr0 set.
1130
1131	/*
1132	* _eval_af will always be called when needed (complex but infrequent):
1133	* - if after inc, af is set when 4 low order bits of op1 are 0
1134	* - if after dec, af is set when 4 low order bits of op1 are 1
1135	* (or 0 after adding 1 as implemented here)
1136	* - if after add/sub/adc/sbb/cmp af is set from sum of 4 LSB of op1
1137	* and 4 LSB of op2 (eventually complemented) plus carry in.
1138	* - other instructions leave AF undefined so the returned value is irrelevant.
1139	* Returned value must be masked with 0x10, since all other bits are undefined.
1140	* There branchless code is perhaps not the most efficient, but quite parallel.
1141	*/
1142	_eval_af: rlwinm r3,flags,0,INCDEC_FIELD
1143	clrlwi r5,op2,28 # 4 LSB of op2
1144	addc r4,flags,flags # carry_in
1145	GET_ADDSUB(r6)
1146	cntlzw r3,r3 # if inc/dec 16..23 else 32
1147	neg r6,r6 # add/sub
1148	clrlwi r4,r3,31 # if dec 1 else 0
1149	xor r5,r5,r6 # conditionally complement
1150	clrlwi r6,op1,28 # 4 LSB of op1
1151	add r4,op2,r4 # op2+(dec ? 1 : 0)
1152	clrlwi r4,r4,28 # 4 LSB of op2+(dec ? 1 : 0)
1153	adde r5,r6,r5 # op1+cy_in+(op2/~op2)
1154	cntlzw r4,r4 # 28..31 if not AF, 32 if set
1155	andc r5,r5,r3 # masked AF from add/sub...
1156	andc r4,r3,r4 # masked AF from inc/dec
1157	or r3,r4,r5
1158	blr
1159
1160	/*
1161	* _eval_above will only be called if ABOVE_IN_CR is not set.
1162	* On return: ZF=cr6[2], CF=cr4[0], ABOVE=cr4[1]
1163	*/
1164	_eval_above: andis. r3,flags,ZF_IN_CR>>16
1165	mflr r0
1166	beql+ _eval_sf_zf
1167	andis. r3,flags,CF_IN_CR>>16
1168	beql+ _eval_cf
1169	mtlr r0
1170	oris flags,flags,ABOVE_IN_CR>>16
1171	crnor ABOVE,ZF,CF
1172	blr
1173
1174	/* _eval_signed may only be called when signed_in_cr is clear ! */
1175	_eval_signed: andis. r3,flags,SF_IN_CR>>16
1176	mflr r0
1177	beql+ _eval_sf_zf
1178	# SF_IN_CR and ZF_IN_CR are set, SIGNED_IN_CR is clear
1179	rlwinm. r3,flags,5,0,1
1180	xoris flags,flags,(SIGNED_IN_CR\|SF_IN_CR)>>16
1181	bngl+ _eval_of
1182	andis. r3,flags,OF_VALUE>>16
1183	mtlr r0
1184	crxor SLT,SF,OF
1185	crnor SGT,SLT,ZF
1186	blr
1187
1188	_eval_flags: mflr r0
1189	bl _eval_cf
1190	li r7,2
1191	rlwimi r7,r3,24,CF86,CF86 # 2 if CF clear, 3 if set
1192	bl _eval_pf
1193	andis. r4,flags,SF_IN_CR>>16
1194	rlwimi r7,r3,32+PF-PF86,PF86,PF86
1195	bl _eval_af
1196	rlwimi r7,r3,0,AF86,AF86
1197	beql+ _eval_sf_zf
1198	mfcr r3
1199	rlwinm. r4,flags,5,0,1
1200	rlwimi r7,r3,0,DF86,SF86
1201	ZF2ZF86(r3,r7)
1202	bngl+ _eval_of
1203	mtlr r0
1204	lis r4,0x0004
1205	lwz r3,eflags(state)
1206	addi r4,r4,0x7000
1207	rlwimi r7,flags,17,OF86,OF86
1208	and r3,r3,r4
1209	or r3,r3,r7
1210	blr
1211
1212	/* Quite simple for real mode, input in r4, returns in r3. */
1213	_segment_load: lwz r5,vbase(state)
1214	rlwinm r3,r4,4,0xffff0 # segment selector * 16
1215	add r3,r3,r5
1216	blr
1217
1218	/* To allow I/O port virtualization if necessary, code for exception in r3,
1219	port number in r4 */
1220	_check_port: lwz r5,ioperm(state)
1221	rlwinm r6,r4,29,0x1fff # 0 to 8kB
1222	lis r0,0xffff
1223	lhbrx r5,r5,r6
1224	clrlwi r6,r4,29 # modulo 8
1225	rlwnm r0,r0,r3,0x0f # 1, 3, or 0xf
1226	slw r0,r0,r6
1227	and. r0,r0,r5
1228	bne- complex
1229	blr
1230	/*
1231	* Instructions are in approximate functional order:
1232	* 1) move, exchange, lea, push/pop, pusha/popa
1233	* 2) cbw/cwde/cwd/cdq, zero/sign extending moves, in/out
1234	* 3) arithmetic: add/sub/adc/sbb/cmp/inc/dec/neg
1235	* 4) logical: and/or/xor/test/not/bt/btc/btr/bts/bsf/bsr
1236	* 5) jump, call, ret
1237	* 6) string instructions and xlat
1238	* 7) rotate/shift/mul/div
1239	* 8) segment register, far jumps, calls and rets, interrupts
1240	* 9) miscellenaous (flags, bcd,...)
1241	*/
1242
1243	#define MEM offset,base
1244	#define REG opreg,state
1245	#define SELECTORS 32
1246	#define SELBASES 64
1247
1248	/* Immediate moves */
1249	movb_imm_reg: rlwinm opreg,opcode,2,28,29; lbz r3,1(eip)
1250	rlwimi opreg,opcode,30,31,31; lbzu opcode,2(eip)
1251	stbx r3,REG; GOTNEXT
1252
1253	movw_imm_reg: lhz r3,1(eip); clrlslwi opreg,opcode,29,2; lbzu opcode,3(eip)
1254	sthx r3,REG; GOTNEXT
1255
1256	movl_imm_reg: lwz r3,1(eip); clrlslwi opreg,opcode,29,2; lbzu opcode,5(eip)
1257	stwx r3,REG; GOTNEXT
1258
1259	movb_imm_mem: lbz r0,1(eip); cmpwi opreg,0
1260	lbzu opcode,2(eip); bne- ud
1261	stbx r0,MEM; GOTNEXT
1262
1263	movw_imm_mem: lhz r0,1(eip); cmpwi opreg,0
1264	lbzu opcode,3(eip); bne- ud
1265	sthx r0,MEM; GOTNEXT
1266
1267	movl_imm_mem: lwz r0,1(eip); cmpwi opreg,0
1268	lbzu opcode,5(eip); bne- ud
1269	stwx r0,MEM; GOTNEXT
1270
1271	/* The special short form moves between memory and al/ax/eax */
1272	movb_al_a32: lwbrx offset,eip,one; lbz r0,AL(state); lbzu opcode,5(eip)
1273	stbx r0,MEM; GOTNEXT
1274
1275	movb_al_a16: lhbrx offset,eip,one; lbz r0,AL(state); lbzu opcode,3(eip)
1276	stbx r0,MEM; GOTNEXT
1277
1278	movw_ax_a32: lwbrx offset,eip,one; lhz r0,AX(state); lbzu opcode,5(eip)
1279	sthx r0,MEM; GOTNEXT
1280
1281	movw_ax_a16: lhbrx offset,eip,one; lhz r0,AX(state); lbzu opcode,3(eip)
1282	sthx r0,MEM; GOTNEXT
1283
1284	movl_eax_a32: lwbrx offset,eip,one; lwz r0,EAX(state); lbzu opcode,5(eip)
1285	stwx r0,MEM; GOTNEXT
1286
1287	movl_eax_a16: lhbrx offset,eip,one; lwz r0,EAX(state); lbzu opcode,3(eip)
1288	stwx r0,MEM; GOTNEXT
1289
1290	movb_a32_al: lwbrx offset,eip,one; lbzu opcode,5(eip); lbzx r0,MEM
1291	stb r0,AL(state); GOTNEXT
1292
1293	movb_a16_al: lhbrx offset,eip,one; lbzu opcode,3(eip); lbzx r0,MEM
1294	stb r0,AL(state); GOTNEXT
1295
1296	movw_a32_ax: lwbrx offset,eip,one; lbzu opcode,5(eip); lhzx r0,MEM
1297	sth r0,AX(state); GOTNEXT
1298
1299	movw_a16_ax: lhbrx offset,eip,one; lbzu opcode,3(eip); lhzx r0,MEM
1300	sth r0,AX(state); GOTNEXT
1301
1302	movl_a32_eax: lwbrx offset,eip,one; lbzu opcode,5(eip); lwzx r0,MEM
1303	stw r0,EAX(state); GOTNEXT
1304
1305	movl_a16_eax: lhbrx offset,eip,one; lbzu opcode,3(eip); lwzx r0,MEM
1306	stw r0,EAX(state); GOTNEXT
1307
1308	/* General purpose move (all are exactly 4 instructions long) */
1309	.align 4
1310	movb_reg_mem: lbzx r0,REG
1311	NEXTBYTE(opcode)
1312	stbx r0,MEM
1313	GOTNEXT
1314
1315	movw_reg_mem: lhzx r0,REG
1316	NEXTBYTE(opcode)
1317	sthx r0,MEM
1318	GOTNEXT
1319
1320	movl_reg_mem: lwzx r0,REG
1321	NEXTBYTE(opcode)
1322	stwx r0,MEM
1323	GOTNEXT
1324
1325	movb_mem_reg: lbzx r0,MEM
1326	NEXTBYTE(opcode)
1327	stbx r0,REG
1328	GOTNEXT
1329
1330	movw_mem_reg: lhzx r0,MEM
1331	NEXTBYTE(opcode)
1332	sthx r0,REG
1333	GOTNEXT
1334
1335	movl_mem_reg: lwzx r0,MEM
1336	NEXTBYTE(opcode)
1337	stwx r0,REG
1338	GOTNEXT
1339
1340	/* short form exchange ax/eax with register */
1341	xchgw_ax_reg: clrlslwi opreg,opcode,29,2
1342	lhz r3,AX(state)
1343	lhzx r4,REG
1344	sthx r3,REG
1345	sth r4,AX(state)
1346	NEXT
1347
1348	xchgl_eax_reg: clrlslwi opreg,opcode,29,2
1349	lwz r3,EAX(state)
1350	lwzx r4,REG
1351	stwx r3,REG
1352	stw r4,EAX(state)
1353	NEXT
1354
1355	/* General exchange (unlocked!) */
1356	xchgb_reg_mem: lbzx r3,MEM
1357	lbzx r4,REG
1358	NEXTBYTE(opcode)
1359	stbx r3,REG
1360	stbx r4,MEM
1361	GOTNEXT
1362
1363	xchgw_reg_mem: lhzx r3,MEM
1364	lhzx r4,REG
1365	sthx r3,REG
1366	sthx r4,MEM
1367	NEXT
1368
1369	xchgl_reg_mem: lwzx r3,MEM
1370	lwzx r4,REG
1371	stwx r3,REG
1372	stwx r4,MEM
1373	NEXT
1374
1375	/* lea, one of the simplest instructions */
1376	leaw: cmpw base,state
1377	beq- ud
1378	sthbrx offset,REG
1379	NEXT
1380
1381	leal: cmpw base,state
1382	beq- ud
1383	stwbrx offset,REG
1384	NEXT
1385
1386	/* Short form pushes and pops */
1387	pushw_sp_reg: li r3,SP
1388	lhbrx r4,state,r3
1389	clrlslwi opreg,opcode,29,2
1390	lhzx r0,REG
1391	addi r4,r4,-2
1392	sthbrx r4,state,r3
1393	clrlwi r4,r4,16
1394	sthx r0,ssb,r4
1395	NEXT
1396
1397	pushl_sp_reg: li r3,SP
1398	lhbrx r4,state,r3
1399	clrlslwi opreg,opcode,29,2
1400	lwzx r0,REG
1401	addi r4,r4,-4
1402	sthbrx r4,state,r3
1403	clrlwi r4,r4,16
1404	stwx r0,ssb,r4
1405	NEXT
1406
1407	popw_sp_reg: li r3,SP
1408	lhbrx r4,state,r3
1409	clrlslwi opreg,opcode,29,2
1410	lhzx r0,ssb,r4
1411	addi r4,r4,2 # order is important in case of pop sp
1412	sthbrx r4,state,r3
1413	sthx r0,REG
1414	NEXT
1415
1416	popl_sp_reg: li r3,SP
1417	lhbrx r4,state,r3
1418	clrlslwi opreg,opcode,29,2
1419	lwzx r0,ssb,r4
1420	addi r4,r4,4
1421	sthbrx r4,state,r3
1422	stwx r0,REG
1423	NEXT
1424
1425	/* Push immediate */
1426	pushw_sp_imm: li r3,SP
1427	lhbrx r4,state,r3
1428	lhz r0,1(eip)
1429	addi r4,r4,-2
1430	sthbrx r4,state,r3
1431	clrlwi r4,r4,16
1432	lbzu opcode,3(eip)
1433	sthx r0,ssb,r4
1434	GOTNEXT
1435
1436	pushl_sp_imm: li r3,SP
1437	lhbrx r4,state,r3
1438	lwz r0,1(eip)
1439	addi r4,r4,-4
1440	sthbrx r4,state,r3
1441	clrlwi r4,r4,16
1442	lbzu opcode,5(eip)
1443	stwx r0,ssb,r4
1444	GOTNEXT
1445
1446	pushw_sp_imm8: li r3,SP
1447	lhbrx r4,state,r3
1448	lhz r0,1(eip)
1449	addi r4,r4,-2
1450	sthbrx r4,state,r3
1451	clrlwi r4,r4,16
1452	lbzu opcode,2(eip)
1453	extsb r0,r0
1454	sthx r0,ssb,r4
1455	GOTNEXT
1456
1457	pushl_sp_imm8: li r3,SP
1458	lhbrx r4,state,r3
1459	lhz r0,1(eip)
1460	addi r4,r4,-4
1461	sthbrx r4,state,r3
1462	clrlwi r4,r4,16
1463	lbzu opcode,2(eip)
1464	extsb r0,r0
1465	stwx r0,ssb,r4
1466	GOTNEXT
1467
1468	/* General push/pop */
1469	pushw_sp: lhbrx r0,MEM
1470	li r3,SP
1471	lhbrx r4,state,r3
1472	addi r4,r4,-2
1473	sthbrx r4,state,r3
1474	clrlwi r4,r4,16
1475	sthbrx r0,r4,ssb
1476	NEXT
1477
1478	pushl_sp: lwbrx r0,MEM
1479	li r3,SP
1480	lhbrx r4,state,r3
1481	addi r4,r4,-4
1482	sthbrx r4,state,r3
1483	clrlwi r4,r4,16
1484	stwbrx r0,r4,ssb
1485	NEXT
1486
1487	/* pop is an exception with 32 bit addressing modes, it is possible
1488	to calculate wrongly the address when esp is used as base. But 16 bit
1489	addressing modes are safe */
1490
1491	popw_sp_a16: cmpw cr1,opreg,0 # first check the opcode
1492	li r3,SP
1493	lhbrx r4,state,r3
1494	bne- cr1,ud
1495	lhzx r0,ssb,r4
1496	addi r4,r4,2
1497	sthx r0,MEM
1498	sthbrx r4,state,r3
1499	NEXT
1500
1501	popl_sp_a16: cmpw cr1,opreg,0
1502	li r3,SP
1503	lhbrx r4,state,r3
1504	bne- cr1,ud
1505	lwzx r0,ssb,r4
1506	addi r4,r4,2
1507	stwx r0,MEM
1508	sthbrx r4,state,r3
1509	NEXT
1510
1511	/* 32 bit addressing modes for pop not implemented for now. */
1512	.equ popw_sp_a32,unimpl
1513	.equ popl_sp_a32,unimpl
1514
1515	/* pusha/popa */
1516	pushaw_sp: li r3,SP
1517	li r0,8
1518	lhbrx r4,r3,state
1519	mtctr r0
1520	addi r5,state,-4
1521	1: addi r4,r4,-2
1522	lhzu r6,4(r5)
1523	clrlwi r4,r4,16
1524	sthx r6,ssb,r4
1525	bdnz 1b
1526	sthbrx r4,r3,state # new sp
1527	NEXT
1528
1529	pushal_sp: li r3,SP
1530	li r0,8
1531	lhbrx r4,r3,state
1532	mtctr r0
1533	addi r5,state,-4
1534	1: addi r4,r4,-4
1535	lwzu r6,4(r5)
1536	clrlwi r4,r4,16
1537	stwx r6,ssb,r4
1538	bdnz 1b
1539	sthbrx r4,r3,state # new sp
1540	NEXT
1541
1542	popaw_sp: li r3,SP
1543	li r0,8
1544	lhbrx r4,state,r3
1545	mtctr r0
1546	addi r5,state,32
1547	1: lhzx r6,ssb,r4
1548	addi r4,r4,2
1549	sthu r6,-4(r5)
1550	clrlwi r4,r4,16
1551	bdnz 1b
1552	sthbrx r4,r3,state # updated sp
1553	NEXT
1554
1555	popal_sp: li r3,SP
1556	lis r0,0xef00 # mask to skip esp
1557	lhbrx r4,state,r3
1558	addi r5,state,32
1559	1: add. r0,r0,r0
1560	lwzx r6,ssb,r4
1561	addi r4,r4,4
1562	stwu r6,-4(r5)
1563	clrlwi r4,r4,16
1564	blt 1b
1565	addi r6,r6,-4
1566	beq 2f
1567	addi r4,r4,4
1568	clrlwi r4,r4,16
1569	b 1b
1570	2: sthbrx r4,state,r3 # updated sp
1571	NEXT
1572
1573	/* Moves with zero or sign extension: first the special cases */
1574	cbw: lbz r3,AL(state)
1575	extsb r3,r3
1576	sthbrx r3,AX,state
1577	NEXT
1578
1579	cwde: lhbrx r3,AX,state
1580	extsh r3,r3
1581	stwbrx r3,EAX,state
1582	NEXT
1583
1584	cwd: lbz r3,AH(state)
1585	extsb r3,r3
1586	srwi r3,r3,8 # get sign bits
1587	sth r3,DX(state)
1588	NEXT
1589
1590	cdq: lwbrx r3,EAX,state
1591	srawi r3,r3,31
1592	stw r3,EDX(state) # byte order unimportant !
1593	NEXT
1594
1595	/* The move with zero or sign extension are special since the source
1596	and destination are not the same size. The register describing the destination
1597	is modified to take this into account. */
1598
1599	movsbw: lbzx r3,MEM
1600	rlwimi opreg,opreg,4,0x10
1601	extsb r3,r3
1602	rlwinm opreg,opreg,0,0x1c
1603	sthbrx r3,REG
1604	NEXT
1605
1606	movsbl: lbzx r3,MEM
1607	rlwimi opreg,opreg,4,0x10
1608	extsb r3,r3
1609	rlwinm opreg,opreg,0,0x1c
1610	stwbrx r3,REG
1611	NEXT
1612
1613	.equ movsww, movw_mem_reg
1614
1615	movswl: lhbrx r3,MEM
1616	extsh r3,r3
1617	stwbrx r3,REG
1618	NEXT
1619
1620	movzbw: lbzx r3,MEM
1621	rlwimi opreg,opreg,4,0x10
1622	rlwinm opreg,opreg,0,0x1c
1623	sthbrx r3,REG
1624	NEXT
1625
1626	movzbl: lbzx r3,MEM
1627	rlwimi opreg,opreg,4,0x10
1628	rlwinm opreg,opreg,0,0x1c
1629	stwbrx r3,REG
1630	NEXT
1631
1632	.equ movzww, movw_mem_reg
1633
1634	movzwl: lhbrx r3,MEM
1635	stwbrx r3,REG
1636	NEXT
1637
1638	/* Byte swapping */
1639	bswap: clrlslwi opreg,opcode,29,2 # extract reg from opcode
1640	lwbrx r0,REG
1641	stwx r0,REG
1642	NEXT
1643
1644	/* Input/output */
1645	inb_port_al: NEXTBYTE(r4)
1646	b 1f
1647	inb_dx_al: li r4,DX
1648	lhbrx r4,r4,state
1649	1: li r3,code_inb
1650	bl _check_port
1651	lwz r3,iobase(state)
1652	lbzx r5,r4,r3
1653	eieio
1654	stb r5,AL(state)
1655	NEXT
1656
1657	inw_port_ax: NEXTBYTE(r4)
1658	b 1f
1659	inw_dx_ax: li r4,DX
1660	lhbrx r4,r4,state
1661	1: li r3,code_inw
1662	bl _check_port
1663	lwz r3,iobase(state)
1664	lhzx r5,r4,r3
1665	eieio
1666	sth r5,AX(state)
1667	NEXT
1668
1669	inl_port_eax: NEXTBYTE(r4)
1670	b 1f
1671	inl_dx_eax: li r4,DX
1672	lhbrx r4,r4,state
1673	1: li r3,code_inl
1674	bl _check_port
1675	lwz r3,iobase(state)
1676	lwzx r5,r4,r3
1677	eieio
1678	stw r5,EAX(state)
1679	NEXT
1680
1681	outb_al_port: NEXTBYTE(r4)
1682	b 1f
1683	outb_al_dx: li r4,DX
1684	lhbrx r4,r4,state
1685	1: li r3,code_outb
1686	bl _check_port
1687	lwz r3,iobase(state)
1688	lbz r5,AL(state)
1689	stbx r5,r4,r3
1690	eieio
1691	NEXT
1692
1693	outw_ax_port: NEXTBYTE(r4)
1694	b 1f
1695	outw_ax_dx: li r4,DX
1696	lhbrx r4,r4,state
1697	1: li r3,code_outw
1698	bl _check_port
1699	lwz r3,iobase(state)
1700	lhz r5,AX(state)
1701	sthx r5,r4,r3
1702	eieio
1703	NEXT
1704
1705	outl_eax_port: NEXTBYTE(r4)
1706	b 1f
1707	outl_eax_dx: li r4,DX
1708	lhbrx r4,r4,state
1709	1: li r3,code_outl
1710	bl _check_port
1711	lwz r4,iobase(state)
1712	lwz r5,EAX(state)
1713	stwx r5,r4,r3
1714	eieio
1715	NEXT
1716
1717
1718	/* Macro used for add and sub */
1719	#define ARITH(op,fl) \
1720	op##b_reg_mem: lbzx op1,MEM; SET_FLAGS(fl(B)); lbzx op2,REG; \
1721	op result,op1,op2; \
1722	stbx result,MEM; NEXT; \
1723	op##w_reg_mem: lhbrx op1,MEM; SET_FLAGS(fl(W)); lhbrx op2,REG; \
1724	op result,op1,op2; \
1725	sthbrx result,MEM; NEXT; \
1726	op##l_reg_mem: lwbrx op1,MEM; SET_FLAGS(fl(L)); lwbrx op2,REG; \
1727	op result,op1,op2; \
1728	stwbrx result,MEM; NEXT; \
1729	op##b_mem_reg: lbzx op2,MEM; SET_FLAGS(fl(B)); lbzx op1,REG; \
1730	op result,op1,op2; \
1731	stbx result,REG; NEXT; \
1732	op##w_mem_reg: lhbrx op2,MEM; SET_FLAGS(fl(W)); lhbrx op1,REG; \
1733	op result,op1,op2; \
1734	sthbrx result,REG; NEXT; \
1735	op##l_mem_reg: lwbrx op2,MEM; SET_FLAGS(fl(L)); lwbrx op1,REG; \
1736	op result,op1,op2; \
1737	stwbrx result,REG; NEXT; \
1738	op##b_imm_al: addi base,state,0; li offset,AL; \
1739	op##b_imm: lbzx op1,MEM; SET_FLAGS(fl(B)); lbz op2,1(eip); \
1740	op result,op1,op2; \
1741	lbzu opcode,2(eip); \
1742	stbx result,MEM; GOTNEXT; \
1743	op##w_imm_ax: addi base,state,0; li offset,AX; \
1744	op##w_imm: lhbrx op1,MEM; SET_FLAGS(fl(W)); lhbrx op2,eip,one; \
1745	op result,op1,op2; \
1746	lbzu opcode,3(eip); \
1747	sthbrx result,MEM; GOTNEXT; \
1748	op##w_imm8: lbz op2,1(eip); SET_FLAGS(fl(W)); lhbrx op1,MEM; \
1749	extsb op2,op2; clrlwi op2,op2,16; \
1750	op result,op1,op2; \
1751	lbzu opcode,2(eip); \
1752	sthbrx result,MEM; GOTNEXT; \
1753	op##l_imm_eax: addi base,state,0; li offset,EAX; \
1754	op##l_imm: lwbrx op1,MEM; SET_FLAGS(fl(L)); lwbrx op2,eip,one; \
1755	op result,op1,op2; lbzu opcode,5(eip); \
1756	stwbrx result,MEM; GOTNEXT; \
1757	op##l_imm8: lbz op2,1(eip); SET_FLAGS(fl(L)); lwbrx op1,MEM; \
1758	extsb op2,op2; lbzu opcode,2(eip); \
1759	op result,op1,op2; \
1760	stwbrx result,MEM; GOTNEXT
1761
1762	ARITH(add, FLAGS_ADD)
1763	ARITH(sub, FLAGS_SUB)
1764
1765	#define adc(result, op1, op2) adde result,op1,op2
1766	#define sbb(result, op1, op2) subfe result,op2,op1
1767
1768	#define ARITH_WITH_CARRY(op, fl) \
1769	op##b_reg_mem: lbzx op1,MEM; bl carryfor##op; lbzx op2,REG; \
1770	ADD_FLAGS(fl(B)); op(result, op1, op2); \
1771	stbx result,MEM; NEXT; \
1772	op##w_reg_mem: lhbrx op1,MEM; bl carryfor##op; lhbrx op2,REG; \
1773	ADD_FLAGS(fl(W)); op(result, op1, op2); \
1774	sthbrx result,MEM; NEXT; \
1775	op##l_reg_mem: lwbrx op1,MEM; bl carryfor##op; lwbrx op2,REG; \
1776	ADD_FLAGS(fl(L)); op(result, op1, op2); \
1777	stwbrx result,MEM; NEXT; \
1778	op##b_mem_reg: lbzx op1,MEM; bl carryfor##op; lbzx op2,REG; \
1779	ADD_FLAGS(fl(B)); op(result, op1, op2); \
1780	stbx result,REG; NEXT; \
1781	op##w_mem_reg: lhbrx op1,MEM; bl carryfor##op; lhbrx op2,REG; \
1782	ADD_FLAGS(fl(W)); op(result, op1, op2); \
1783	sthbrx result,REG; NEXT; \
1784	op##l_mem_reg: lwbrx op1,MEM; bl carryfor##op; lwbrx op2,REG; \
1785	ADD_FLAGS(fl(L)); op(result, op1, op2); \
1786	stwbrx result,REG; NEXT; \
1787	op##b_imm_al: addi base,state,0; li offset,AL; \
1788	op##b_imm: lbzx op1,MEM; bl carryfor##op; lbz op2,1(eip); \
1789	ADD_FLAGS(fl(B)); lbzu opcode,2(eip); op(result, op1, op2); \
1790	stbx result,MEM; GOTNEXT; \
1791	op##w_imm_ax: addi base,state,0; li offset,AX; \
1792	op##w_imm: lhbrx op1,MEM; bl carryfor##op; lhbrx op2,eip,one; \
1793	ADD_FLAGS(fl(W)); lbzu opcode,3(eip); op(result, op1, op2); \
1794	sthbrx result,MEM; GOTNEXT; \
1795	op##w_imm8: lbz op2,1(eip); bl carryfor##op; lhbrx op1,MEM; \
1796	extsb op2,op2; ADD_FLAGS(fl(W)); clrlwi op2,op2,16; \
1797	lbzu opcode,2(eip); op(result, op1, op2); \
1798	sthbrx result,MEM; GOTNEXT; \
1799	op##l_imm_eax: addi base,state,0; li offset,EAX; \
1800	op##l_imm: lwbrx op1,MEM; bl carryfor##op; lwbrx op2,eip,one; \
1801	ADD_FLAGS(fl(L)); lbzu opcode,5(eip); op(result, op1, op2); \
1802	stwbrx result,MEM; GOTNEXT; \
1803	op##l_imm8: lbz op2,1(eip); SET_FLAGS(fl(L)); lwbrx op1,MEM; \
1804	extsb op2,op2; lbzu opcode,2(eip); \
1805	op(result, op1, op2); \
1806	stwbrx result,MEM; GOTNEXT
1807
1808	carryforadc: addc r3,flags,flags # CF_IN to xer[ca]
1809	RES2CF(r4) # get 8 or 16 bit carry
1810	subfe r3,result,op1 # generate PPC carry for
1811	CF_ROTCNT(r5) # preceding operation
1812	addze r3,r4 # 32 bit carry in LSB
1813	CF_POL(r4,23) # polarity
1814	rlwnm r3,r3,r5,0x100 # shift carry there
1815	xor flags,r4,r3 # CF86 ? 0x100 : 0
1816	addic r4,r3,0xffffff00 # set xer[ca]
1817	rlwinm flags,r3,23,CF_IN
1818	blr
1819
1820	ARITH_WITH_CARRY(adc, FLAGS_ADD)
1821
1822	/* for sbb the input carry must be the complement of the x86 carry */
1823	carryforsbb: addc r3,flags,flags # CF_IN to xer[ca]
1824	RES2CF(r4) # 8/16 bit carry from result
1825	subfe r3,result,op1
1826	CF_ROTCNT(r5)
1827	addze r3,r4
1828	CF_POL(r4,23)
1829	rlwnm r3,r3,r5,0x100
1830	eqv flags,r4,r3 # CF86 ? 0xfffffeff:0xffffffff
1831	addic r4,r3,1 # set xer[ca]
1832	rlwinm flags,r3,23,CF_IN # keep only the carry
1833	blr
1834
1835	ARITH_WITH_CARRY(sbb, FLAGS_SBB)
1836
1837	cmpb_reg_mem: lbzx op1,MEM
1838	SET_FLAGS(FLAGS_CMP(B))
1839	lbzx op2,REG
1840	extsb r3,op1
1841	cmplw cr4,op1,op2
1842	extsb r4,op2
1843	sub result,op1,op2
1844	cmpw cr6,r3,r4
1845	NEXT
1846
1847	cmpw_reg_mem: lhbrx op1,MEM
1848	SET_FLAGS(FLAGS_CMP(W))
1849	lhbrx op2,REG
1850	extsh r3,op1
1851	cmplw cr4,op1,op2
1852	extsh r4,op2
1853	sub result,op1,op2
1854	cmpw cr6,r3,r4
1855	NEXT
1856
1857	cmpl_reg_mem: lwbrx op1,MEM
1858	SET_FLAGS(FLAGS_CMP(L))
1859	lwbrx op2,REG
1860	cmplw cr4,op1,op2
1861	sub result,op1,op2
1862	cmpw cr6,op1,op2
1863	NEXT
1864
1865	cmpb_mem_reg: lbzx op2,MEM
1866	SET_FLAGS(FLAGS_CMP(B))
1867	lbzx op1,REG
1868	extsb r4,op2
1869	cmplw cr4,op1,op2
1870	extsb r3,op1
1871	sub result,op1,op2
1872	cmpw cr6,r3,r4
1873	NEXT
1874
1875	cmpw_mem_reg: lhbrx op2,MEM
1876	SET_FLAGS(FLAGS_CMP(W))
1877	lhbrx op1,REG
1878	extsh r4,op2
1879	cmplw cr4,op1,op2
1880	extsh r3,op1
1881	sub result,op1,op2
1882	cmpw cr6,r3,r4
1883	NEXT
1884
1885	cmpl_mem_reg: lwbrx op2,MEM
1886	SET_FLAGS(FLAGS_CMP(L))
1887	lwbrx op1,REG
1888	cmpw cr6,op1,op2
1889	sub result,op1,op2
1890	cmplw cr4,op1,op2
1891	NEXT
1892
1893	cmpb_imm_al: addi base,state,0
1894	li offset,AL
1895	cmpb_imm: lbzx op1,MEM
1896	SET_FLAGS(FLAGS_CMP(B))
1897	lbz op2,1(eip)
1898	extsb r3,op1
1899	cmplw cr4,op1,op2
1900	lbzu opcode,2(eip)
1901	extsb r4,op2
1902	sub result,op1,op2
1903	cmpw cr6,r3,r4
1904	GOTNEXT
1905
1906	cmpw_imm_ax: addi base,state,0
1907	li offset,AX
1908	cmpw_imm: lhbrx op1,MEM
1909	SET_FLAGS(FLAGS_CMP(W))
1910	lhbrx op2,eip,one
1911	extsh r3,op1
1912	cmplw cr4,op1,op2
1913	lbzu opcode,3(eip)
1914	extsh r4,op2
1915	sub result,op1,op2
1916	cmpw cr6,r3,r4
1917	GOTNEXT
1918
1919	cmpw_imm8: lbz op2,1(eip)
1920	SET_FLAGS(FLAGS_CMP(W))
1921	lhbrx op1,MEM
1922	extsb r4,op2
1923	extsh r3,op1
1924	lbzu opcode,2(eip)
1925	clrlwi op2,r4,16
1926	cmpw cr6,r3,r4
1927	sub result,op1,op2
1928	cmplw cr4,op1,op2
1929	GOTNEXT
1930
1931	cmpl_imm_eax: addi base,state,0
1932	li offset,EAX
1933	cmpl_imm: lwbrx op1,MEM
1934	SET_FLAGS(FLAGS_CMP(L))
1935	lwbrx op2,eip,one
1936	cmpw cr6,op1,op2
1937	lbzu opcode,5(eip)
1938	sub result,op1,op2
1939	cmplw cr4,op1,op2
1940	GOTNEXT
1941
1942	cmpl_imm8: lbz op2,1(eip)
1943	SET_FLAGS(FLAGS_CMP(L))
1944	lwbrx op1,MEM
1945	extsb op2,op2
1946	lbzu opcode,2(eip)
1947	cmpw cr6,op1,op2
1948	sub result,op1,op2
1949	cmplw cr4,op1,op2
1950	GOTNEXT
1951
1952	/* Increment and decrement */
1953	incb: lbzx op2,MEM
1954	INC_FLAGS(B)
1955	addi op2,op2,1
1956	stbx op2,MEM
1957	NEXT
1958
1959	incw_reg: clrlslwi opreg,opcode,29,2 # extract reg from opcode
1960	lhbrx op2,REG
1961	INC_FLAGS(W)
1962	addi op2,op2,1
1963	sthbrx op2,REG
1964	NEXT
1965
1966	incw: lhbrx op2,MEM
1967	INC_FLAGS(W)
1968	addi op2,op2,1
1969	sthbrx op2,MEM
1970	NEXT
1971
1972	incl_reg: clrlslwi opreg,opcode,29,2
1973	lwbrx op2,REG
1974	INC_FLAGS(L)
1975	addi op2,op2,1
1976	sthbrx op2,REG
1977	NEXT
1978
1979	incl: lwbrx op2,MEM
1980	INC_FLAGS(L)
1981	addi op2,op2,1
1982	stwbrx op2,MEM
1983	NEXT
1984
1985	decb: lbzx op2,MEM
1986	DEC_FLAGS(B)
1987	addi op2,op2,-1
1988	stbx op2,MEM
1989	NEXT
1990
1991	decw_reg: clrlslwi opreg,opcode,29,2 # extract reg from opcode
1992	lhbrx op2,REG
1993	DEC_FLAGS(W)
1994	addi op2,op2,-1
1995	sthbrx op2,REG
1996	NEXT
1997
1998	decw: lhbrx op2,MEM
1999	DEC_FLAGS(W)
2000	addi op2,op2,-1
2001	sthbrx op2,MEM
2002	NEXT
2003
2004	decl_reg: clrlslwi opreg,opcode,29,2
2005	lwbrx op2,REG
2006	DEC_FLAGS(L)
2007	addi op2,op2,-1
2008	sthbrx op2,REG
2009	NEXT
2010
2011	decl: lwbrx op2,MEM
2012	DEC_FLAGS(L)
2013	addi op2,op2,-1
2014	stwbrx op2,MEM
2015	NEXT
2016
2017	negb: lbzx op2,MEM
2018	SET_FLAGS(FLAGS_SUB(B))
2019	neg result,op2
2020	li op1,0
2021	stbx result,MEM
2022	NEXT
2023
2024	negw: lhbrx op2,MEM
2025	SET_FLAGS(FLAGS_SUB(W))
2026	neg result,op2
2027	li op1,0
2028	sthbrx r0,MEM
2029	NEXT
2030
2031	negl: lwbrx op2,MEM
2032	SET_FLAGS(FLAGS_SUB(L))
2033	subfic result,op2,0
2034	li op1,0
2035	stwbrx result,MEM
2036	NEXT
2037
2038	/* Macro used to generate code for OR/AND/XOR */
2039	#define LOGICAL(op) \
2040	op##b_reg_mem: lbzx op1,MEM; SET_FLAGS(FLAGS_LOG(B)); lbzx op2,REG; \
2041	op result,op1,op2; \
2042	stbx result,MEM; NEXT; \
2043	op##w_reg_mem: lhbrx op1,MEM; SET_FLAGS(FLAGS_LOG(W)); lhbrx op2,REG; \
2044	op result,op1,op2; \
2045	sthbrx result,MEM; NEXT; \
2046	op##l_reg_mem: lwbrx op1,MEM; SET_FLAGS(FLAGS_LOG(L)); lwbrx op2,REG; \
2047	op result,op1,op2; \
2048	stwbrx result,MEM; NEXT; \
2049	op##b_mem_reg: lbzx op1,MEM; SET_FLAGS(FLAGS_LOG(B)); lbzx op2,REG; \
2050	op result,op1,op2; \
2051	stbx result,REG; NEXT; \
2052	op##w_mem_reg: lhbrx op2,MEM; SET_FLAGS(FLAGS_LOG(W)); lhbrx op1,REG; \
2053	op result,op1,op2; \
2054	sthbrx result,REG; NEXT; \
2055	op##l_mem_reg: lwbrx op2,MEM; SET_FLAGS(FLAGS_LOG(L)); lwbrx op1,REG; \
2056	op result,op1,op2; \
2057	stwbrx result,REG; NEXT; \
2058	op##b_imm_al: addi base,state,0; li offset,AL; \
2059	op##b_imm: lbzx op1,MEM; SET_FLAGS(FLAGS_LOG(B)); lbz op2,1(eip); \
2060	op result,op1,op2; lbzu opcode,2(eip); \
2061	stbx result,MEM; GOTNEXT; \
2062	op##w_imm_ax: addi base,state,0; li offset,AX; \
2063	op##w_imm: lhbrx op1,MEM; SET_FLAGS(FLAGS_LOG(W)); lhbrx op2,eip,one; \
2064	op result,op1,op2; lbzu opcode,3(eip); \
2065	sthbrx result,MEM; GOTNEXT; \
2066	op##w_imm8: lbz op2,1(eip); SET_FLAGS(FLAGS_LOG(W)); lhbrx op1,MEM; \
2067	extsb op2,op2; lbzu opcode,2(eip); \
2068	op result,op1,op2; \
2069	sthbrx result,MEM; GOTNEXT; \
2070	op##l_imm_eax: addi base,state,0; li offset,EAX; \
2071	op##l_imm: lwbrx op1,MEM; SET_FLAGS(FLAGS_LOG(L)); lwbrx op2,eip,one; \
2072	op result,op1,op2; lbzu opcode,5(eip); \
2073	stwbrx result,MEM; GOTNEXT; \
2074	op##l_imm8: lbz op2,1(eip); SET_FLAGS(FLAGS_LOG(L)); lwbrx op1,MEM; \
2075	extsb op2,op2; lbzu opcode,2(eip); \
2076	op result,op1,op2; \
2077	stwbrx result,MEM; GOTNEXT
2078
2079	LOGICAL(or)
2080
2081	LOGICAL(and)
2082
2083	LOGICAL(xor)
2084
2085	testb_reg_mem: lbzx op1,MEM
2086	SET_FLAGS(FLAGS_TEST(B))
2087	lbzx op2,REG
2088	and result,op1,op2
2089	extsb r3,result
2090	cmpwi cr6,r3,0
2091	NEXT
2092
2093	testw_reg_mem: lhbrx op1,MEM
2094	SET_FLAGS(FLAGS_TEST(W))
2095	lhbrx op2,REG
2096	and result,op1,op2
2097	extsh r3,result
2098	cmpwi cr6,r3,0
2099	NEXT
2100
2101	testl_reg_mem: lwbrx r3,MEM
2102	SET_FLAGS(FLAGS_TEST(L))
2103	lwbrx r4,REG
2104	and result,op1,op2
2105	cmpwi cr6,result,0
2106	NEXT
2107
2108	testb_imm_al: addi base,state,0
2109	li offset,AL
2110	testb_imm: lbzx op1,MEM
2111	SET_FLAGS(FLAGS_TEST(B))
2112	lbz op2,1(eip)
2113	and result,op1,op2
2114	lbzu opcode,2(eip)
2115	extsb r3,result
2116	cmpwi cr6,r3,0
2117	GOTNEXT
2118
2119	testw_imm_ax: addi base,state,0
2120	li offset,AX
2121	testw_imm: lhbrx op1,MEM
2122	SET_FLAGS(FLAGS_TEST(W))
2123	lhbrx op2,eip,one
2124	and result,op1,op2
2125	lbzu opcode,3(eip)
2126	extsh r3,result
2127	cmpwi cr6,r3,0
2128	GOTNEXT
2129
2130	testl_imm_eax: addi base,state,0
2131	li offset,EAX
2132	testl_imm: lwbrx op1,MEM
2133	SET_FLAGS(FLAGS_TEST(L))
2134	lwbrx op2,eip,one
2135	and result,r3,r4
2136	lbzu opcode,5(eip)
2137	cmpwi cr6,result,0
2138	GOTNEXT
2139
2140	/* Not does not affect flags */
2141	notb: lbzx r3,MEM
2142	xori r3,r3,255
2143	stbx r3,MEM
2144	NEXT
2145
2146	notw: lhzx r3,MEM
2147	xori r3,r3,65535
2148	sthx r3,MEM
2149	NEXT
2150
2151	notl: lwzx r3,MEM
2152	not r3,r3
2153	stwx r3,MEM
2154	NEXT
2155
2156	boundw: lhbrx r4,REG
2157	li r3,code_bound
2158	lhbrx r5,MEM
2159	addi offset,offset,2
2160	extsh r4,r4
2161	lhbrx r6,MEM
2162	extsh r5,r5
2163	cmpw r4,r5
2164	extsh r6,r6
2165	blt- complex
2166	cmpw r4,r6
2167	ble+ nop
2168	b complex
2169
2170	boundl: lwbrx r4,REG
2171	li r3,code_bound
2172	lwbrx r5,MEM
2173	addi offset,offset,4
2174	lwbrx r6,MEM
2175	cmpw r4,r5
2176	blt- complex
2177	cmpw r4,r6
2178	ble+ nop
2179	b complex
2180
2181	/* Bit test and modify instructions */
2182
2183	/* Common routine: bit index in op2, returns memory value in r3, mask in op2,
2184	and of mask and value in op1. CF flag is set as with 32 bit add when bit is
2185	non zero since result (which is cleared) will be less than op1, and in cr4,
2186	all other flags are undefined from Intel doc. Here OF and SF are cleared
2187	and ZF is set as a side effect of result being cleared. */
2188	_setup_bitw: cmpw base,state
2189	SET_FLAGS(FLAGS_BTEST)
2190	extsh op2,op2
2191	beq- 1f
2192	srawi r4,op2,4
2193	add offset,offset,r4
2194	1: clrlwi op2,op2,28 # true bit index
2195	lhbrx r3,MEM
2196	slw op2,one,op2 # build mask
2197	li result,0 # implicitly sets CF
2198	and op1,r3,op2 # if result<op1
2199	cmplw cr4,result,op1 # sets CF in cr4
2200	blr
2201
2202	_setup_bitl: cmpw base,state
2203	SET_FLAGS(FLAGS_BTEST)
2204	beq- 1f
2205	srawi r4,op2,5
2206	add offset,offset,r4
2207	1: lwbrx r3,MEM
2208	rotlw op2,one,op2 # build mask
2209	li result,0
2210	and op1,r3,op2
2211	cmplw cr4,result,op1
2212	blr
2213
2214	/* Immediate forms bit tests are not frequent since logical are often faster */
2215	btw_imm: NEXTBYTE(op2)
2216	b 1f
2217	btw_reg_mem: lhbrx op2,REG
2218	1: bl _setup_bitw
2219	NEXT
2220
2221	btl_imm: NEXTBYTE(op2)
2222	b 1f
2223	btl_reg_mem: lhbrx op2,REG
2224	1: bl _setup_bitl
2225	NEXT
2226
2227	btcw_imm: NEXTBYTE(op2)
2228	b 1f
2229	btcw_reg_mem: lhbrx op2,REG
2230	1: bl _setup_bitw
2231	xor r3,r3,op2
2232	sthbrx r3,MEM
2233	NEXT
2234
2235	btcl_imm: NEXTBYTE(op2)
2236	b 1f
2237	btcl_reg_mem: lhbrx op2,REG
2238	1: bl _setup_bitl
2239	xor r3,r3,op2
2240	stwbrx result,MEM
2241	NEXT
2242
2243	btrw_imm: NEXTBYTE(op2)
2244	b 1f
2245	btrw_reg_mem: lhbrx op2,REG
2246	1: bl _setup_bitw
2247	andc r3,r3,op2
2248	sthbrx r3,MEM
2249	NEXT
2250
2251	btrl_imm: NEXTBYTE(op2)
2252	b 1f
2253	btrl_reg_mem: lhbrx op2,REG
2254	1: bl _setup_bitl
2255	andc r3,r3,op2
2256	stwbrx r3,MEM
2257	NEXT
2258
2259	btsw_imm: NEXTBYTE(op2)
2260	b 1f
2261	btsw_reg_mem: lhbrx op2,REG
2262	1: bl _setup_bitw
2263	or r3,r3,op2
2264	sthbrx r3,MEM
2265	NEXT
2266
2267	btsl_imm: NEXTBYTE(op2)
2268	b 1f
2269	btsl_reg_mem: lhbrx op2,REG
2270	1: bl _setup_bitl
2271	or r3,r3,op2
2272	stwbrx r3,MEM
2273	NEXT
2274
2275	/* Bit string search instructions, only ZF is defined after these, and the
2276	result value is not defined when the bit field is zero. */
2277	bsfw: lhbrx result,MEM
2278	SET_FLAGS(FLAGS_BSRCH(W))
2279	neg r3,result
2280	cmpwi cr6,result,0 # sets ZF
2281	and r3,r3,result # keep only LSB
2282	cntlzw r3,r3
2283	subfic r3,r3,31
2284	sthbrx r3,REG
2285	NEXT
2286
2287	bsfl: lwbrx result,MEM
2288	SET_FLAGS(FLAGS_BSRCH(L))
2289	neg r3,result
2290	cmpwi cr6,result,0 # sets ZF
2291	and r3,r3,result # keep only LSB
2292	cntlzw r3,r3
2293	subfic r3,r3,31
2294	stwbrx r3,REG
2295	NEXT
2296
2297	bsrw: lhbrx result,MEM
2298	SET_FLAGS(FLAGS_BSRCH(W))
2299	cntlzw r3,result
2300	cmpwi cr6,result,0
2301	subfic r3,r3,31
2302	sthbrx r3,REG
2303	NEXT
2304
2305	bsrl: lwbrx result,MEM
2306	SET_FLAGS(FLAGS_BSRCH(L))
2307	cntlzw r3,result
2308	cmpwi cr6,result,0
2309	subfic r3,r3,31
2310	stwbrx r3,REG
2311	NEXT
2312
2313	/* Unconditional jumps, first the indirect than relative */
2314	jmpw: lhbrx eip,MEM
2315	lbzux opcode,eip,csb
2316	GOTNEXT
2317
2318	jmpl: lwbrx eip,MEM
2319	lbzux opcode,eip,csb
2320	GOTNEXT
2321
2322	sjmp_w: lbz r3,1(eip)
2323	sub eip,eip,csb
2324	addi eip,eip,2 # EIP after instruction
2325	extsb r3,r3
2326	add eip,eip,r3
2327	clrlwi eip,eip,16 # module 64k
2328	lbzux opcode,eip,csb
2329	GOTNEXT
2330
2331	jmp_w: lhbrx r3,eip,one # eip now off by 3
2332	sub eip,eip,csb
2333	addi r3,r3,3 # compensate
2334	add eip,eip,r3
2335	clrlwi eip,eip,16
2336	lbzux opcode,eip,csb
2337	GOTNEXT
2338
2339	sjmp_l: lbz r3,1(eip)
2340	addi eip,eip,2
2341	extsb r3,r3
2342	lbzux opcode,eip,r3
2343	GOTNEXT
2344
2345	jmp_l: lwbrx r3,eip,one # Simple
2346	addi eip,eip,5
2347	lbzux opcode,eip,r3
2348	GOTNEXT
2349
2350	/* The conditional jumps: although it should not happen,
2351	byte relative jumps (sjmp) may wrap around in 16 bit mode */
2352
2353	#define NOTTAKEN_S lbzu opcode,2(eip); GOTNEXT
2354	#define NOTTAKEN_W lbzu opcode,3(eip); GOTNEXT
2355	#define NOTTAKEN_L lbzu opcode,5(eip); GOTNEXT
2356
2357	#define CONDJMP(cond, eval, flag) \
2358	sj##cond##_w: EVAL_##eval; bt flag,sjmp_w; NOTTAKEN_S; \
2359	j##cond##_w: EVAL_##eval; bt flag,jmp_w; NOTTAKEN_W; \
2360	sj##cond##_l: EVAL_##eval; bt flag,sjmp_l; NOTTAKEN_S; \
2361	j##cond##_l: EVAL_##eval; bt flag,jmp_l; NOTTAKEN_L; \
2362	sjn##cond##_w: EVAL_##eval; bf flag,sjmp_w; NOTTAKEN_S; \
2363	jn##cond##_w: EVAL_##eval; bf flag,jmp_w; NOTTAKEN_W; \
2364	sjn##cond##_l: EVAL_##eval; bf flag,sjmp_l; NOTTAKEN_S; \
2365	jn##cond##_l: EVAL_##eval; bf flag,jmp_l; NOTTAKEN_L
2366
2367	CONDJMP(o, OF, OF)
2368	CONDJMP(c, CF, CF)
2369	CONDJMP(z, ZF, ZF)
2370	CONDJMP(a, ABOVE, ABOVE)
2371	CONDJMP(s, SF, SF)
2372	CONDJMP(p, PF, PF)
2373	CONDJMP(g, SIGNED, SGT)
2374	CONDJMP(l, SIGNED, SLT)
2375
2376	jcxz_w: lhz r3,CX(state); cmpwi r3,0; beq- sjmp_w; NOTTAKEN_S
2377	jcxz_l: lhz r3,CX(state); cmpwi r3,0; beq- sjmp_l; NOTTAKEN_S
2378	jecxz_w: lwz r3,ECX(state); cmpwi r3,0; beq- sjmp_w; NOTTAKEN_S
2379	jecxz_l: lwz r3,ECX(state); cmpwi r3,0; beq- sjmp_l; NOTTAKEN_S
2380
2381	/* Note that loop is somewhat strange, the data size attribute gives
2382	the size of eip, and the address size whether the counter is cx or ecx.
2383	This is the same for jcxz/jecxz. */
2384
2385	loopw_w: li opreg,CX
2386	lhbrx r0,REG
2387	sub. r0,r0,one
2388	sthbrx r0,REG
2389	bne+ sjmp_w
2390	NOTTAKEN_S
2391
2392	loopl_w: li opreg,ECX
2393	lwbrx r0,REG
2394	sub. r0,r0,one
2395	stwbrx r0,REG
2396	bne+ sjmp_w
2397	NOTTAKEN_S
2398
2399	loopw_l: li opreg,CX
2400	lhbrx r0,REG
2401	sub. r0,r0,one
2402	sthbrx r0,REG
2403	bne+ sjmp_l
2404	NOTTAKEN_S
2405
2406	loopl_l: li opreg,ECX
2407	lwbrx r0,REG
2408	sub. r0,r0,one
2409	stwbrx r0,REG
2410	bne+ sjmp_l
2411	NOTTAKEN_S
2412
2413	loopzw_w: li opreg,CX
2414	lhbrx r0,REG
2415	EVAL_ZF
2416	sub. r0,r0,one
2417	sthbrx r0,REG
2418	bf ZF,1f
2419	bne+ sjmp_w
2420	1: NOTTAKEN_S
2421
2422	loopzl_w: li opreg,ECX
2423	lwbrx r0,REG
2424	EVAL_ZF
2425	sub. r3,r3,one
2426	stwbrx r3,REG
2427	bf ZF,1f
2428	bne+ sjmp_w
2429	1: NOTTAKEN_S
2430
2431	loopzw_l: li opreg,CX
2432	lhbrx r0,REG
2433	EVAL_ZF
2434	sub. r0,r0,one
2435	sthbrx r0,REG
2436	bf ZF,1f
2437	bne+ sjmp_l
2438	1: NOTTAKEN_S
2439
2440	loopzl_l: li opreg,ECX
2441	lwbrx r0,REG
2442	EVAL_ZF
2443	sub. r0,r0,one
2444	stwbrx r0,REG
2445	bf ZF,1f
2446	bne+ sjmp_l
2447	1: NOTTAKEN_S
2448
2449	loopnzw_w: li opreg,CX
2450	lhbrx r0,REG
2451	EVAL_ZF
2452	sub. r0,r0,one
2453	sthbrx r0,REG
2454	bt ZF,1f
2455	bne+ sjmp_w
2456	1: NOTTAKEN_S
2457
2458	loopnzl_w: li opreg,ECX
2459	lwbrx r0,REG
2460	EVAL_ZF
2461	sub. r0,r0,one
2462	stwbrx r0,REG
2463	bt ZF,1f
2464	bne+ sjmp_w
2465	1: NOTTAKEN_S
2466
2467	loopnzw_l: li opreg,CX
2468	lhbrx r0,REG
2469	EVAL_ZF
2470	sub. r0,r0,one
2471	sthbrx r0,REG
2472	bt ZF,1f
2473	bne+ sjmp_l
2474	1: NOTTAKEN_S
2475
2476	loopnzl_l: li opreg,ECX
2477	lwbrx r0,REG
2478	EVAL_ZF
2479	sub. r0,r0,one
2480	stwbrx r0,REG
2481	bt ZF,1f
2482	bne+ sjmp_l
2483	1: NOTTAKEN_S
2484
2485	/* Memory indirect calls are rare enough to limit code duplication */
2486	callw_sp_mem: lhbrx r3,MEM
2487	sub r4,eip,csb
2488	addi r4,r4,1 # r4 is now return address
2489	b 1f
2490	.equ calll_sp_mem, unimpl
2491
2492	callw_sp: lhbrx r3,eip,one
2493	sub r4,eip,csb
2494	addi r4,r4,3 # r4 is return address
2495	add r3,r4,r3
2496	1: clrlwi eip,r3,16
2497	li r5,SP
2498	lhbrx r6,state,r5 # get sp
2499	addi r6,r6,-2
2500	lbzux opcode,eip,csb
2501	sthbrx r6,state,r5 # update sp
2502	clrlwi r6,r6,16
2503	sthbrx r4,ssb,r6 # push return address
2504	GOTNEXT
2505	.equ calll_sp, unimpl
2506
2507	retw_sp_imm: li opreg,SP
2508	lhbrx r4,REG
2509	lhbrx r6,eip,one
2510	addi r5,r4,2
2511	lhbrx eip,ssb,r4
2512	lbzux opcode,eip,csb
2513	add r5,r5,r6
2514	sthbrx r5,REG
2515	GOTNEXT
2516
2517	.equ retl_sp_imm, unimpl
2518
2519	retw_sp: li opreg,SP
2520	lhbrx r4,REG
2521	addi r5,r4,2
2522	lhbrx eip,ssb,r4
2523	lbzux opcode,eip,csb
2524	sthbrx r5,REG
2525	GOTNEXT
2526
2527	.equ retl_sp, unimpl
2528
2529	/* Enter is a mess, and the description in Intel documents is actually wrong
2530	* in most revisions (all PPro/PII I have but the old Pentium is Ok) !
2531	*/
2532
2533	enterw_sp: lhbrx r0,eip,one # Stack space to allocate
2534	li opreg,SP
2535	lhbrx r3,REG # SP
2536	li r7,BP
2537	lbzu r4,3(eip) # nesting level
2538	addi r3,r3,-2
2539	lhbrx r5,state,r7 # Original BP
2540	clrlwi r3,r3,16
2541	sthbrx r5,ssb,r3 # Push BP
2542	andi. r4,r4,31 # modulo 32 and test
2543	mr r6,r3 # Save frame pointer to temp
2544	beq 3f
2545	mtctr r4 # iterate level-1 times
2546	b 2f
2547	1: addi r5,r5,-2 # copy list of frame pointers
2548	clrlwi r5,r5,16
2549	lhzx r4,ssb,r5
2550	addi r3,r3,-2
2551	clrlwi r3,r3,16
2552	sthx r4,ssb,r3
2553	2: bdnz 1b
2554	addi r3,r3,-2 # save current frame pointer
2555	clrlwi r3,r3,16
2556	sthbrx r6,ssb,r3
2557	3: sthbrx r6,state,r7 # New BP
2558	sub r3,r3,r0
2559	sthbrx r3,REG # Save new stack pointer
2560	NEXT
2561
2562	.equ enterl_sp, unimpl
2563
2564	leavew_sp: li opreg,BP
2565	lhbrx r3,REG # Stack = BP
2566	addi r4,r3,2 #
2567	lhzx r3,ssb,r3
2568	li opreg,SP
2569	sthbrx r4,REG # New Stack
2570	sth r3,BP(state) # Popped BP
2571	NEXT
2572
2573	.equ leavel_sp, unimpl
2574
2575	/* String instructions: first a generic setup routine, which exits early
2576	if there is a repeat prefix with a count of 0 */
2577	#define STRINGSRC base,offset
2578	#define STRINGDST esb,opreg
2579
2580	_setup_stringw: li offset,SI #
2581	rlwinm. r3,opcode,19,0,1 # lt=repnz, gt= repz, eq none
2582	li opreg,DI
2583	lhbrx offset,state,offset # load si
2584	li r3,1 # no repeat
2585	lhbrx opreg,state,opreg # load di
2586	beq 1f # no repeat
2587	li r3,CX
2588	lhbrx r3,state,r3 # load CX
2589	cmpwi r3,0
2590	beq nop # early exit here !
2591	1: mtctr r3 # ctr=CX or 1
2592	li r7,1 # stride
2593	bflr+ DF
2594	li r7,-1 # change stride sign
2595	blr
2596
2597	/* Ending routine to update all changed registers (goes directly to NEXT) */
2598	_finish_strw: li r4,SI
2599	sthbrx offset,state,r4 # update si
2600	li r4,DI
2601	sthbrx opreg,state,r4 # update di
2602	beq nop
2603	mfctr r3
2604	li r4,CX
2605	sthbrx r3,state,r4 # update cx
2606	NEXT
2607
2608
2609	lodsb_a16: bl _setup_stringw
2610	1: lbzx r0,STRINGSRC # [rep] lodsb
2611	add offset,offset,r7
2612	clrlwi offset,offset,16
2613	bdnz 1b
2614	stb r0,AL(state)
2615	b _finish_strw
2616
2617	lodsw_a16: bl _setup_stringw
2618	slwi r7,r7,1
2619	1: lhzx r0,STRINGSRC # [rep] lodsw
2620	add offset,offset,r7
2621	clrlwi offset,offset,16
2622	bdnz 1b
2623	sth r0,AX(state)
2624	b _finish_strw
2625
2626	lodsl_a16: bl _setup_stringw
2627	slwi r7,r7,2
2628	1: lwzx r0,STRINGSRC # [rep] lodsl
2629	add offset,offset,r7
2630	clrlwi offset,offset,16
2631	bdnz 1b
2632	stw r0,EAX(state)
2633	b _finish_strw
2634
2635	stosb_a16: bl _setup_stringw
2636	lbz r0,AL(state)
2637	1: stbx r0,STRINGDST # [rep] stosb
2638	add opreg,opreg,r7
2639	clrlwi opreg,opreg,16
2640	bdnz 1b
2641	b _finish_strw
2642
2643	stosw_a16: bl _setup_stringw
2644	lhz r0,AX(state)
2645	slwi r7,r7,1
2646	1: sthx r0,STRINGDST # [rep] stosw
2647	add opreg,opreg,r7
2648	clrlwi opreg,opreg,16
2649	bdnz 1b
2650	b _finish_strw
2651
2652	stosl_a16: bl _setup_stringw
2653	lwz r0,EAX(state)
2654	slwi r7,r7,2
2655	1: stwx r0,STRINGDST # [rep] stosl
2656	add opreg,opreg,r7
2657	clrlwi opreg,opreg,16
2658	bdnz 1b
2659	b _finish_strw
2660
2661	movsb_a16: bl _setup_stringw
2662	1: lbzx r0,STRINGSRC # [rep] movsb
2663	add offset,offset,r7
2664	stbx r0,STRINGDST
2665	clrlwi offset,offset,16
2666	add opreg,opreg,r7
2667	clrlwi opreg,opreg,16
2668	bdnz 1b
2669	b _finish_strw
2670
2671	movsw_a16: bl _setup_stringw
2672	slwi r7,r7,1
2673	1: lhzx r0,STRINGSRC # [rep] movsw
2674	add offset,offset,r7
2675	sthx r0,STRINGDST
2676	clrlwi offset,offset,16
2677	add opreg,opreg,r7
2678	clrlwi opreg,opreg,16
2679	bdnz 1b
2680	b _finish_strw
2681
2682	movsl_a16: bl _setup_stringw
2683	slwi r7,r7,2
2684	1: lwzx r0,STRINGSRC # [rep] movsl
2685	add offset,offset,r7
2686	stwx r0,STRINGDST
2687	clrlwi offset,offset,16
2688	add opreg,opreg,r7
2689	clrlwi opreg,opreg,16
2690	bdnz 1b
2691	b _finish_strw
2692
2693	/* At least on a Pentium, repeated string I/O instructions check for
2694	access port permission even if count is 0 ! So the order of the check is not
2695	important. */
2696	insb_a16: li r4,DX
2697	li r3,code_insb_a16
2698	lhbrx r4,state,r4
2699	bl _check_port
2700	bl _setup_stringw
2701	lwz base,iobase(state)
2702	1: lbzx r0,base,r4 # [rep] insb
2703	eieio
2704	stbx r0,STRINGDST
2705	add opreg,opreg,r7
2706	clrlwi opreg,opreg,16
2707	bdnz 1b
2708	b _finish_strw
2709
2710	insw_a16: li r4,DX
2711	li r3,code_insw_a16
2712	lhbrx r4,state,r4
2713	bl _check_port
2714	bl _setup_stringw
2715	lwz base,iobase(state)
2716	slwi r7,r7,1
2717	1: lhzx r0,base,r4 # [rep] insw
2718	eieio
2719	sthx r0,STRINGDST
2720	add opreg,opreg,r7
2721	clrlwi opreg,opreg,16
2722	bdnz 1b
2723	b _finish_strw
2724
2725	insl_a16: li r4,DX
2726	li r3,code_insl_a16
2727	lhbrx r4,state,r4
2728	bl _check_port
2729	bl _setup_stringw
2730	lwz base,iobase(state)
2731	slwi r7,r7,2
2732	1: lwzx r0,base,r4 # [rep] insl
2733	eieio
2734	stwx r0,STRINGDST
2735	add opreg,opreg,r7
2736	clrlwi opreg,opreg,16
2737	bdnz 1b
2738	b _finish_strw
2739
2740	outsb_a16: li r4,DX
2741	li r3,code_outsb_a16
2742	lhbrx r4,state,r4
2743	bl _check_port
2744	bl _setup_stringw
2745	lwz r6,iobase(state)
2746	1: lbzx r0,STRINGSRC # [rep] outsb
2747	add offset,offset,r7
2748	stbx r0,r6,r4
2749	clrlwi offset,offset,16
2750	eieio
2751	bdnz 1b
2752	b _finish_strw
2753
2754	outsw_a16: li r4,DX
2755	li r3,code_outsw_a16
2756	lhbrx r4,state,r4
2757	bl _check_port
2758	bl _setup_stringw
2759	li r5,DX
2760	lwz r6,iobase(state)
2761	slwi r7,r7,1
2762	1: lhzx r0,STRINGSRC # [rep] outsw
2763	add offset,offset,r7
2764	sthx r0,r6,r4
2765	clrlwi offset,offset,16
2766	eieio
2767	bdnz 1b
2768	b _finish_strw
2769
2770	outsl_a16: li r4,DX
2771	li r3,code_outsl_a16
2772	lhbrx r4,state,r4
2773	bl _check_port
2774	bl _setup_stringw
2775	lwz r6,iobase(state)
2776	slwi r7,r7,2
2777	1: lwzx r0,STRINGSRC # [rep] outsl
2778	add offset,offset,r7
2779	stwx r0,r6,r4
2780	clrlwi offset,offset,16
2781	eieio
2782	bdnz 1b
2783	b _finish_strw
2784
2785	cmpsb_a16: bl _setup_stringw
2786	SET_FLAGS(FLAGS_CMP(B))
2787	blt 3f # repnz prefix
2788	1: lbzx op1,STRINGSRC # [repz] cmpsb
2789	add offset,offset,r7
2790	lbzx op2,STRINGDST
2791	add opreg,opreg,r7
2792	cmplw cr4,op1,op2
2793	clrlwi offset,offset,16
2794	clrlwi opreg,opreg,16
2795	bdnzt CF+2,1b
2796	2: extsb r3,op1
2797	extsb r4,op2
2798	cmpw cr6,r3,r4
2799	sub result,op1,op2
2800	b _finish_strw
2801
2802	3: lbzx op1,STRINGSRC # repnz cmpsb
2803	add offset,offset,r7
2804	lbzx op2,STRINGDST
2805	add opreg,opreg,r7
2806	cmplw cr4,op1,op2
2807	clrlwi offset,offset,16
2808	clrlwi opreg,opreg,16
2809	bdnzf CF+2,3b
2810	b 2b
2811
2812	cmpsw_a16: bl _setup_stringw
2813	SET_FLAGS(FLAGS_CMP(W))
2814	slwi r7,r7,1
2815	blt 3f # repnz prefix
2816	1: lhbrx op1,STRINGSRC # [repz] cmpsb
2817	add offset,offset,r7
2818	lhbrx op2,STRINGDST
2819	add opreg,opreg,r7
2820	cmplw cr4,op1,op2
2821	clrlwi offset,offset,16
2822	clrlwi opreg,opreg,16
2823	bdnzt CF+2,1b
2824	2: extsh r3,op1
2825	extsh r4,op2
2826	cmpw cr6,r3,r4
2827	sub result,op1,op2
2828	b _finish_strw
2829
2830	3: lhbrx op1,STRINGSRC # repnz cmpsw
2831	add offset,offset,r7
2832	lhbrx op2,STRINGDST
2833	add opreg,opreg,r7
2834	cmplw cr4,op1,op2
2835	clrlwi offset,offset,16
2836	clrlwi opreg,opreg,16
2837	bdnzf CF+2,3b
2838	b 2b
2839
2840	cmpsl_a16: bl _setup_stringw
2841	SET_FLAGS(FLAGS_CMP(L))
2842	slwi r7,r7,2
2843	blt 3f # repnz prefix
2844	1: lwbrx op1,STRINGSRC # [repz] cmpsl
2845	add offset,offset,r7
2846	lwbrx op2,STRINGDST
2847	add opreg,opreg,r7
2848	cmplw cr4,op1,op2
2849	clrlwi offset,offset,16
2850	clrlwi opreg,opreg,16
2851	bdnzt CF+2,1b
2852	2: cmpw cr6,op1,op2
2853	sub result,op1,op2
2854	b _finish_strw
2855
2856	3: lwbrx op1,STRINGSRC # repnz cmpsl
2857	add offset,offset,r7
2858	lwbrx op2,STRINGDST
2859	add opreg,opreg,r7
2860	cmplw cr4,op1,op2
2861	clrlwi offset,offset,16
2862	clrlwi opreg,opreg,16
2863	bdnzf CF+2,3b
2864	b 2b
2865
2866	scasb_a16: bl _setup_stringw
2867	lbzx op1,AL,state # AL
2868	SET_FLAGS(FLAGS_CMP(B))
2869	bgt 3f # repz prefix
2870	1: lbzx op2,STRINGDST # [repnz] scasb
2871	add opreg,opreg,r7
2872	cmplw cr4,op1,op2
2873	clrlwi opreg,opreg,16
2874	bdnzf CF+2,1b
2875	2: extsb r3,op1
2876	extsb r4,op2
2877	cmpw cr6,r3,r4
2878	sub result,op1,op2
2879	b _finish_strw
2880
2881	3: lbzx op2,STRINGDST # repz scasb
2882	add opreg,opreg,r7
2883	cmplw cr4,op1,op2
2884	clrlwi opreg,opreg,16
2885	bdnzt CF+2,3b
2886	b 2b
2887
2888	scasw_a16: bl _setup_stringw
2889	lhbrx op1,AX,state
2890	SET_FLAGS(FLAGS_CMP(W))
2891	slwi r7,r7,1
2892	bgt 3f # repz prefix
2893	1: lhbrx op2,STRINGDST # [repnz] scasw
2894	add opreg,opreg,r7
2895	cmplw cr4,op1,op2
2896	clrlwi opreg,opreg,16
2897	bdnzf CF+2,1b
2898	2: extsh r3,op1
2899	extsh r4,op2
2900	cmpw cr6,r3,r4
2901	sub result,op1,op2
2902	b _finish_strw
2903
2904	3: lhbrx op2,STRINGDST # repz scasw
2905	add opreg,opreg,r7
2906	cmplw cr4,op1,op2
2907	clrlwi opreg,opreg,16
2908	bdnzt CF+2,3b
2909	b 2b
2910
2911	scasl_a16: bl _setup_stringw
2912	lwbrx op1,EAX,state
2913	SET_FLAGS(FLAGS_CMP(L))
2914	slwi r7,r7,2
2915	bgt 3f # repz prefix
2916	1: lwbrx op2,STRINGDST # [repnz] scasl
2917	add opreg,opreg,r7
2918	cmplw cr4,op1,op2
2919	clrlwi opreg,opreg,16
2920	bdnzf CF+2,1b
2921	2: cmpw cr6,op1,op2
2922	sub result,op1,op2
2923	b _finish_strw
2924
2925	3: lwbrx op2,STRINGDST # repz scasl
2926	add opreg,opreg,r7
2927	cmplw cr4,op1,op2
2928	clrlwi opreg,opreg,16
2929	bdnzt CF+2,3b
2930	b 2b
2931
2932	.equ lodsb_a32, unimpl
2933	.equ lodsw_a32, unimpl
2934	.equ lodsl_a32, unimpl
2935	.equ stosb_a32, unimpl
2936	.equ stosw_a32, unimpl
2937	.equ stosl_a32, unimpl
2938	.equ movsb_a32, unimpl
2939	.equ movsw_a32, unimpl
2940	.equ movsl_a32, unimpl
2941	.equ insb_a32, unimpl
2942	.equ insw_a32, unimpl
2943	.equ insl_a32, unimpl
2944	.equ outsb_a32, unimpl
2945	.equ outsw_a32, unimpl
2946	.equ outsl_a32, unimpl
2947	.equ cmpsb_a32, unimpl
2948	.equ cmpsw_a32, unimpl
2949	.equ cmpsl_a32, unimpl
2950	.equ scasb_a32, unimpl
2951	.equ scasw_a32, unimpl
2952	.equ scasl_a32, unimpl
2953
2954	xlatb_a16: li offset,BX
2955	lbz r3,AL(state)
2956	lhbrx offset,offset,state
2957	add r3,r3,base
2958	lbzx r3,r3,offset
2959	stb r3,AL(state)
2960	NEXT
2961
2962	.equ xlatb_a32, unimpl
2963
2964	/*
2965	* Shift and rotates: note the oddity that rotates do not affect SF/ZF/AF/PF
2966	* but shifts do. Also testing has indicated that rotates with a count of zero
2967	* do not affect any flag. The documentation specifies this for shifts but
2968	* is more obscure for rotates. The overflow flag setting is only specified
2969	* when count is 1, otherwise OF is undefined which simplifies emulation.
2970	*/
2971
2972	/*
2973	* The rotates through carry are among the most difficult instructions,
2974	* they are implemented as a shift of 2*n+some bits depending on case.
2975	* First the left rotates through carry.
2976	*/
2977
2978	/* Byte rcl is performed on 18 bits (17 actually used) in a single register */
2979	rclb_imm: NEXTBYTE(r3)
2980	b 1f
2981	rclb_cl: lbz r3,CL(state)
2982	b 1f
2983	rclb_1: li r3,1
2984	1: lbzx r0,MEM
2985	andi. r3,r3,31 # count%32
2986	addc r4,flags,flags # CF_IN->xer[ca]
2987	RES2CF(r6)
2988	subfe r4,result,op1
2989	mulli r5,r3,29 # 29=ceil(256/9)
2990	CF_ROTCNT(r7)
2991	addze r6,r6
2992	CF_POL_INSERT(r0,23)
2993	srwi r5,r5,8 # count/9
2994	rlwnm r6,r6,r7,0x100
2995	xor r0,r0,r6 # (23)0:CF:data8
2996	rlwimi r5,r5,3,26,28 # 9*(count/9)
2997	rlwimi r0,r0,23,0,7 # CF:(data8):(14)0:CF:data8
2998	sub r3,r3,r5 # count%9
2999	beq- nop # no flags changed if count 0
3000	ROTATE_FLAGS
3001	rlwnm r0,r0,r3,0x000001ff # (23)0:NewCF:Result8
3002	rlwimi flags,r0,19,CF_VALUE
3003	stbx r0,MEM
3004	rlwimi flags,r0,18,OF_XOR
3005	NEXT
3006
3007	/* Word rcl is performed on 33 bits (CF:data16:CF:(15 MSB of data16) */
3008	rclw_imm: NEXTBYTE(r3)
3009	b 1f
3010	rclw_cl: lbz r3,CL(state)
3011	b 1f
3012	rclw_1: li r3,1
3013	1: lhbrx r0,MEM
3014	andi. r3,r3,31 # count=count%32
3015	addc r4,flags,flags
3016	RES2CF(r6)
3017	subfe r4,result,op1
3018	addi r5,r3,15 # modulo 17: >=32 if >=17
3019	CF_ROTCNT(r7)
3020	addze r6,r6
3021	addi r7,r7,8
3022	CF_POL_INSERT(r0,15)
3023	srwi r5,r5,5 # count/17
3024	rlwnm r6,r6,r7,0x10000
3025	rlwimi r5,r5,4,27,27 # 17*(count/17)
3026	xor r0,r0,r6 # (15)0:CF:data16
3027	sub r3,r3,r5 # count%17
3028	rlwinm r4,r0,15,0xffff0000 # CF:(15 MSB of data16):(16)0
3029	slw r0,r0,r3 # New carry and MSBs
3030	rlwnm r4,r4,r3,16,31 # New LSBs
3031	beq- nop # no flags changed if count 0
3032	ROTATE_FLAGS
3033	add r0,r0,r4 # result
3034	rlwimi flags,r0,11,CF_VALUE
3035	sthbrx r0,MEM
3036	rlwimi flags,r0,10,OF_XOR
3037	NEXT
3038
3039	/* Longword rcl only needs 64 bits because the maximum rotate count is 31 ! */
3040	rcll_imm: NEXTBYTE(r3)
3041	b 1f
3042	rcll_cl: lbz r3,CL(state)
3043	b 1f
3044	rcll_1: li r3,1
3045	1: lwbrx r0,MEM
3046	andi. r3,r3,31 # count=count%32
3047	addc r4,r4,flags # ~XER[CA]
3048	RES2CF(r6)
3049	subfe r4,result,op1
3050	CF_ROTCNT(r7)
3051	addze r6,r6
3052	srwi r4,r0,1 # 0:(31 MSB of data32)
3053	addi r7,r7,23
3054	CF_POL_INSERT(r4,0)
3055	rlwnm r6,r6,r7,0,0
3056	beq- nop # no flags changed if count 0
3057	subfic r5,r3,32
3058	xor r4,r4,r6
3059	ROTATE_FLAGS
3060	slw r0,r0,r3 # New MSBs
3061	srw r5,r4,r5 # New LSBs
3062	rlwnm r4,r4,r3,0,0 # New Carry
3063	add r0,r0,r5 # result
3064	rlwimi flags,r4,28,CF_VALUE
3065	rlwimi flags,r0,27,OF_XOR
3066	stwbrx r0,MEM
3067	NEXT
3068
3069	/* right rotates through carry are even worse because PPC only has a left
3070	rotate instruction. Somewhat tough when combined with modulo 9, 17, or
3071	33 operation and the rules of OF and CF flag settings. */
3072	/* Byte rcr is performed on 17 bits */
3073	rcrb_imm: NEXTBYTE(r3)
3074	b 1f
3075	rcrb_cl: lbz r3,CL(state)
3076	b 1f
3077	rcrb_1: li r3,1
3078	1: lbzx r0,MEM
3079	andi. r3,r3,31 # count%32
3080	addc r4,flags,flags # cf_in->xer[ca]
3081	RES2CF(r6)
3082	mulli r5,r3,29 # 29=ceil(256/9)
3083	subfe r4,result,op1
3084	CF_ROTCNT(r7)
3085	addze r6,r6
3086	CF_POL_INSERT(r0,23)
3087	srwi r5,r5,8 # count/9
3088	rlwimi r0,r0,9,0x0001fe00 # (15)0:data8:0:data8
3089	rlwnm r6,r6,r7,0x100
3090	rlwimi r5,r5,3,26,28 # 9*(count/9)
3091	xor r0,r0,r6 # (15)0:data8:CF:data8
3092	sub r3,r3,r5 # count%9
3093	beq- nop # no flags changed if count 0
3094	ROTATE_FLAGS
3095	srw r0,r0,r3 # (23)junk:NewCF:Result8
3096	rlwimi flags,r0,19,CF_VALUE\|OF_XOR
3097	stbx r0,MEM
3098	NEXT
3099
3100	/* Word rcr is a 33 bit right shift with a quirk, because the 33rd bit
3101	is only needed when the rotate count is 16 and rotating left or right
3102	by 16 a 32 bit quantity is the same ! */
3103	rcrw_imm: NEXTBYTE(r3)
3104	b 1f
3105	rcrw_cl: lbz r3,CL(state)
3106	b 1f
3107	rcrw_1: li r3,1
3108	1: lhbrx r0,MEM
3109	andi. r3,r3,31 # count%32
3110	addc r4,flags,flags # cf_in->xer[ca]
3111	RES2CF(r6)
3112	subfe r4,result,op1
3113	addi r5,r3,15 # >=32 if >=17
3114	CF_ROTCNT(r7)
3115	addze r6,r6
3116	addi r7,r7,8
3117	CF_POL_INSERT(r0,15)
3118	srwi r5,r5,5 # count/17
3119	rlwnm r6,r6,r7,0x10000
3120	rlwinm r7,r0,16,0x01 # MSB of data16
3121	rlwimi r0,r0,17,0xfffe0000 # (15 MSB of data16):0:data16
3122	rlwimi r5,r5,4,27,27 # 17*(count/17)
3123	xor r0,r0,r6 # (15 MSB of data16):CF:data16
3124	sub r3,r3,r5 # count%17
3125	beq- nop # no flags changed if count 0
3126	srw r0,r0,r3 # shift right
3127	rlwnm r7,r7,r3,0x10000 # just in case count=16
3128	ROTATE_FLAGS
3129	add r0,r0,r7 # junk15:NewCF:result16
3130	rlwimi flags,r0,11,CF_VALUE\|OF_XOR
3131	sthbrx r0,MEM
3132	NEXT
3133
3134	/* Longword rcr need only 64 bits since the rotate count is limited to 31 */
3135	rcrl_imm: NEXTBYTE(r3)
3136	b 1f
3137	rcrl_cl: lbz r3,CL(state)
3138	b 1f
3139	rcrl_1: li r3,1
3140	1: lwbrx r0,MEM
3141	andi. r3,r3,31 # count%32
3142	addc r4,flags,flags
3143	RES2CF(r6)
3144	subfe r4,result,op1
3145	CF_ROTCNT(r7)
3146	slwi r4,r0,1 # (31MSB of data32):0
3147	addze r6,r6
3148	addi r7,r7,24
3149	CF_POL_INSERT(r4,31)
3150	rlwnm r6,r6,r7,0x01
3151	beq- nop # no flags changed if count 0
3152	subfic r7,r3,32
3153	xor r4,r4,r6
3154	srw r0,r0,r3 # Result LSB
3155	slw r5,r4,r7 # Result MSB
3156	srw r4,r4,r3 # NewCF in LSB
3157	add r0,r0,r5 # result
3158	rlwimi flags,r4,27,CF_VALUE
3159	stwbrx r0,MEM
3160	rlwimi flags,r0,27,OF_XOR
3161	NEXT
3162
3163	/* After the rotates through carry, normal rotates are so simple ! */
3164	rolb_imm: NEXTBYTE(r3)
3165	b 1f
3166	rolb_cl: lbz r3,CL(state)
3167	b 1f
3168	rolb_1: li r3,1
3169	1: lbzx r0,MEM
3170	andi. r4,r3,31 # count%32 == 0 ?
3171	clrlwi r3,r3,29 # count%8
3172	rlwimi r0,r0,24,0xff000000 # replicate for shift in
3173	beq- nop # no flags changed if count 0
3174	ROTATE_FLAGS
3175	rotlw r0,r0,r3
3176	rlwimi flags,r0,27,CF_VALUE # New CF
3177	stbx r0,MEM
3178	rlwimi flags,r0,26,OF_XOR # New OF (CF xor MSB)
3179	NEXT
3180
3181	rolw_imm: NEXTBYTE(r3)
3182	b 1f
3183	rolw_cl: lbz r3,CL(state)
3184	b 1f
3185	rolw_1: li r3,1
3186	1: lhbrx r0,MEM
3187	andi. r3,r3,31
3188	rlwimi r0,r0,16,0,15 # duplicate
3189	beq- nop # no flags changed if count 0
3190	ROTATE_FLAGS
3191	rotlw r0,r0,r3 # result word duplicated
3192	rlwimi flags,r0,27,CF_VALUE # New CF
3193	sthbrx r0,MEM
3194	rlwimi flags,r0,26,OF_XOR # New OF (CF xor MSB)
3195	NEXT
3196
3197	roll_imm: NEXTBYTE(r3)
3198	b 1f
3199	roll_cl: lbz r3,CL(state)
3200	b 1f
3201	roll_1: li r3,1
3202	1: lwbrx r0,MEM
3203	andi. r3,r3,31
3204	beq- nop # no flags changed if count 0
3205	ROTATE_FLAGS
3206	rotlw r0,r0,r3 # result
3207	rlwimi flags,r0,27,CF_VALUE # New CF
3208	stwbrx r0,MEM
3209	rlwimi flags,r0,26,OF_XOR # New OF (CF xor MSB)
3210	NEXT
3211
3212	rorb_imm: NEXTBYTE(r3)
3213	b 1f
3214	rorb_cl: lbz r3,CL(state)
3215	b 1f
3216	rorb_1: li r3,1
3217	1: lbzx r0,MEM
3218	andi. r4,r3,31 # count%32 == 0 ?
3219	clrlwi r3,r3,29 # count%8
3220	rlwimi r0,r0,8,0x0000ff00 # replicate for shift in
3221	beq- nop # no flags changed if count 0
3222	ROTATE_FLAGS
3223	srw r0,r0,r3
3224	rlwimi flags,r0,20,CF_VALUE
3225	stbx r0,MEM
3226	rlwimi flags,r0,19,OF_XOR
3227	NEXT
3228
3229	rorw_imm: NEXTBYTE(r3)
3230	b 1f
3231	rorw_cl: lbz r3,CL(state)
3232	b 1f
3233	rorw_1: li r3,1
3234	1: lhbrx r0,MEM
3235	andi. r4,r3,31
3236	clrlwi r3,r3,28 # count %16
3237	rlwimi r0,r0,16,0xffff0000 # duplicate
3238	beq- nop # no flags changed if count 0
3239	ROTATE_FLAGS
3240	srw r0,r0,r3 # junk16:result16
3241	rlwimi flags,r0,12,CF_VALUE
3242	sthbrx r0,MEM
3243	rlwimi flags,r0,11,OF_XOR
3244	NEXT
3245
3246	rorl_imm: NEXTBYTE(r3)
3247	b 1f
3248	rorl_cl: lbz r3,CL(state)
3249	b 1f
3250	rorl_1: li r3,1
3251	1: lwbrx r0,MEM
3252	andi. r4,r3,31
3253	neg r3,r3
3254	beq- nop # no flags changed if count 0
3255	ROTATE_FLAGS
3256	rotlw r0,r0,r3 # result
3257	rlwimi flags,r0,28,CF_VALUE
3258	stwbrx r0,MEM
3259	rlwimi flags,r0,27,OF_XOR
3260	NEXT
3261
3262	/* Right arithmetic shifts: they clear OF whenever count!=0 */
3263	#define SAR_FLAGS CF_ZERO\|OF_ZERO\|RESL
3264	sarb_imm: NEXTBYTE(r3)
3265	b 1f
3266	sarb_cl: lbz r3,CL(state)
3267	b 1f
3268	sarb_1: li r3,1
3269	1: lbzx r4,MEM
3270	andi. r3,r3,31
3271	addi r5,r3,-1
3272	extsb r4,r4
3273	beq- nop # no flags changed if count 0
3274	SET_FLAGS(SAR_FLAGS)
3275	sraw result,r4,r3
3276	srw r5,r4,r5
3277	stbx result,MEM
3278	rlwimi flags,r5,27,CF_VALUE
3279	NEXT
3280
3281	sarw_imm: NEXTBYTE(r3)
3282	b 1f
3283	sarw_cl: lbz r3,CL(state)
3284	b 1f
3285	sarw_1: li r3,1
3286	1: lhbrx r4,MEM
3287	andi. r3,r3,31
3288	addi r5,r3,-1
3289	extsh r4,r4
3290	beq- nop # no flags changed if count 0
3291	SET_FLAGS(SAR_FLAGS)
3292	sraw result,r4,r3
3293	srw r5,r4,r5
3294	sthbrx result,MEM
3295	rlwimi flags,r5,27,CF_VALUE
3296	NEXT
3297
3298	sarl_imm: NEXTBYTE(r3)
3299	b 1f
3300	sarl_cl: lbz r3,CL(state)
3301	b 1f
3302	sarl_1: li r3,1
3303	1: lwbrx r4,MEM
3304	andi. r3,r3,31
3305	addi r5,r3,-1
3306	beq- nop # no flags changed if count 0
3307	SET_FLAGS(SAR_FLAGS)
3308	sraw result,r4,r3
3309	srw r5,r4,r5
3310	stwbrx result,MEM
3311	rlwimi flags,r5,27,CF_VALUE
3312	NEXT
3313
3314	/* Left shifts are quite easy: they use the flag mechanism of add */
3315	shlb_imm: NEXTBYTE(r3)
3316	b 1f
3317	shlb_cl: lbz r3,CL(state)
3318	b 1f
3319	shlb_1: li r3,1
3320	1: andi. r3,r3,31
3321	beq- nop # no flags changed if count 0
3322	lbzx op1,MEM
3323	SET_FLAGS(FLAGS_ADD(B))
3324	slw result,op1,r3
3325	addi op2,op1,0 # for OF computation only !
3326	stbx result,MEM
3327	NEXT
3328
3329	shlw_imm: NEXTBYTE(r3)
3330	b 1f
3331	shlw_cl: lbz r3,CL(state)
3332	b 1f
3333	shlw_1: li r3,1
3334	1: andi. r3,r3,31
3335	beq- nop # no flags changed if count 0
3336	lhbrx op1,MEM
3337	SET_FLAGS(FLAGS_ADD(W))
3338	slw result,op1,r3
3339	addi op2,op1,0 # for OF computation only !
3340	sthbrx result,MEM
3341	NEXT
3342
3343	/* That one may be wrong */
3344	shll_imm: NEXTBYTE(r3)
3345	b 1f
3346	shll_cl: lbz r3,CL(state)
3347	b 1f
3348	shll_1: li r3,1
3349	1: andi. r3,r3,31
3350	beq- nop # no flags changed if count 0
3351	lwbrx op1,MEM
3352	addi r4,r3,-1
3353	SET_FLAGS(FLAGS_ADD(L))
3354	slw result,op1,r3
3355	addi op2,op1,0 # for OF computation only !
3356	slw op1,op1,r4 # for CF computation
3357	stwbrx result,MEM
3358	NEXT
3359
3360	/* Right shifts are quite complex, because of funny flag rules ! */
3361	shrb_imm: NEXTBYTE(r3)
3362	b 1f
3363	shrb_cl: lbz r3,CL(state)
3364	b 1f
3365	shrb_1: li r3,1
3366	1: andi. r3,r3,31
3367	beq- nop # no flags changed if count 0
3368	lbzx op1,MEM
3369	addi r4,r3,-1
3370	SET_FLAGS(FLAGS_SHR(B))
3371	srw result,op1,r3
3372	srw r4,op1,r4
3373	li op2,-1 # for OF computation only !
3374	stbx result,MEM
3375	rlwimi flags,r4,27,CF_VALUE # Set CF
3376	NEXT
3377
3378	shrw_imm: NEXTBYTE(r3)
3379	b 1f
3380	shrw_cl: lbz r3,CL(state)
3381	b 1f
3382	shrw_1: li r3,1
3383	1: andi. r3,r3,31
3384	beq- nop # no flags changed if count 0
3385	lhbrx op1,MEM
3386	addi r4,r3,-1
3387	SET_FLAGS(FLAGS_SHR(W))
3388	srw result,op1,r3
3389	srw r4,op1,r4
3390	li op2,-1 # for OF computation only !
3391	sthbrx result,MEM
3392	rlwimi flags,r4,27,CF_VALUE # Set CF
3393	NEXT
3394
3395	shrl_imm: NEXTBYTE(r3)
3396	b 1f
3397	shrl_cl: lbz r3,CL(state)
3398	b 1f
3399	shrl_1: li r3,1
3400	1: andi. r3,r3,31
3401	beq- nop # no flags changed if count 0
3402	lwbrx op1,MEM
3403	addi r4,r3,-1
3404	SET_FLAGS(FLAGS_SHR(L))
3405	srw result,op1,r3
3406	srw r4,op1,r4
3407	li op2,-1 # for OF computation only !
3408	stwbrx result,MEM
3409	rlwimi flags,r4,27,CF_VALUE # Set CF
3410	NEXT
3411
3412	/* Double length shifts, shldw uses FLAGS_ADD for simplicity */
3413	shldw_imm: NEXTBYTE(r3)
3414	b 1f
3415	shldw_cl: lbz r3,CL(state)
3416	1: andi. r3,r3,31
3417	beq- nop
3418	lhbrx op1,MEM
3419	SET_FLAGS(FLAGS_ADD(W))
3420	lhbrx op2,REG
3421	rlwimi op1,op2,16,0,15 # op2:op1
3422	addi op2,op1,0
3423	rotlw result,op1,r3
3424	sthbrx result,MEM
3425	NEXT
3426
3427	shldl_imm: NEXTBYTE(r3)
3428	b 1f
3429	shldl_cl: lbz r3,CL(state)
3430	1: andi. r3,r3,31
3431	beq- nop
3432	lwbrx op1,MEM
3433	SET_FLAGS(FLAGS_DBLSH(L))
3434	lwbrx op2,REG
3435	subfic r4,r3,32
3436	slw result,op1,r3
3437	srw r4,op2,r4
3438	rotlw r3,op1,r3
3439	or result,result,r4
3440	addi op2,op1,0
3441	rlwimi flags,r3,27,CF_VALUE
3442	stwbrx result,MEM
3443	NEXT
3444
3445	shrdw_imm: NEXTBYTE(r3)
3446	b 1f
3447	shrdw_cl: lbz r3,CL(state)
3448	1: andi. r3,r3,31
3449	beq- nop
3450	lhbrx op1,MEM
3451	SET_FLAGS(FLAGS_DBLSH(W))
3452	lhbrx op2,REG
3453	addi r4,r3,-1
3454	rlwimi op1,op2,16,0,15 # op2:op1
3455	addi op2,op1,0
3456	srw result,op1,r3
3457	srw r4,op1,r4
3458	sthbrx result,MEM
3459	rlwimi flags,r4,27,CF_VALUE
3460	NEXT
3461
3462	shrdl_imm: NEXTBYTE(r3)
3463	b 1f
3464	shrdl_cl: lbz r3,CL(state)
3465	1: andi. r3,r3,31
3466	beq- nop
3467	lwbrx op1,MEM
3468	SET_FLAGS(FLAGS_DBLSH(L))
3469	lwbrx op2,REG
3470	subfic r4,r3,32
3471	srw result,op1,r3
3472	addi r3,r3,-1
3473	slw r4,op2,r4
3474	srw r3,op1,r3
3475	or result,result,r4
3476	addi op2,op1,0
3477	rlwimi flags,r3,27,CF_VALUE
3478	stwbrx result,MEM
3479	NEXT
3480
3481	/* One operand multiplies: with result double the operand size, unsigned */
3482	mulb: lbzx op2,MEM
3483	lbz op1,AL(state)
3484	mullw result,op1,op2
3485	SET_FLAGS(FLAGS_MUL)
3486	subfic r3,result,255
3487	sthbrx result,AX,state
3488	rlwimi flags,r3,0,CF_VALUE\|OF_VALUE
3489	NEXT
3490
3491	mulw: lhbrx op2,MEM
3492	lhbrx op1,AX,state
3493	mullw result,op1,op2
3494	SET_FLAGS(FLAGS_MUL)
3495	li r4,DX
3496	srwi r3,result,16
3497	sthbrx result,AX,state
3498	neg r5,r3
3499	sthbrx r3,r4,state # DX
3500	rlwimi flags,r5,0,CF_VALUE\|OF_VALUE
3501	NEXT
3502
3503	mull: lwbrx op2,MEM
3504	lwbrx op1,EAX,state
3505	mullw result,op1,op2
3506	mulhwu. r3,op1,op2
3507	SET_FLAGS(FLAGS_MUL)
3508	stwbrx result,EAX,state
3509	li r4,EDX
3510	stwbrx r3,r4,state
3511	beq+ nop
3512	oris flags,flags,(CF_SET\|OF_SET)>>16
3513	NEXT
3514
3515	/* One operand multiplies: with result double the operand size, signed */
3516	imulb: lbzx op2,MEM
3517	extsb op2,op2
3518	lbz op1,AL(state)
3519	extsb op1,op1
3520	mullw result,op1,op2
3521	SET_FLAGS(FLAGS_MUL)
3522	extsb r3,result
3523	sthbrx result,AX,state
3524	cmpw r3,result
3525	beq+ nop
3526	oris flags,flags,(CF_SET\|OF_SET)>>16
3527	NEXT
3528
3529	imulw: lhbrx op2,MEM
3530	extsh op2,op2
3531	lhbrx op1,AX,state
3532	extsh op1,op1
3533	mullw result,op1,op2
3534	SET_FLAGS(FLAGS_MUL)
3535	li r3,DX
3536	extsh r4,result
3537	srwi r5,result,16
3538	sthbrx result,AX,state
3539	cmpw r4,result
3540	sthbrx r5,r3,state
3541	beq+ nop
3542	oris flags,flags,(CF_SET\|OF_SET)>>16
3543	NEXT
3544
3545	imull: lwbrx op2,MEM
3546	SET_FLAGS(FLAGS_MUL)
3547	lwbrx op1,EAX,state
3548	li r3,EDX
3549	mulhw r4,op1,op2
3550	mullw result,op1,op2
3551	stwbrx r4,r3,state
3552	srawi r3,result,31
3553	cmpw r3,r4
3554	beq+ nop
3555	oris flags,flags,(CF_SET\|OF_SET)>>16
3556	NEXT
3557
3558	/* Other multiplies */
3559	imulw_mem_reg: lhbrx op2,REG
3560	extsh op2,op2
3561	b 1f
3562
3563	imulw_imm: NEXTWORD(op2)
3564	extsh op2,op2
3565	b 1f
3566
3567	imulw_imm8: NEXTBYTE(op2)
3568	extsb op2,op2
3569	1: lhbrx op1,MEM
3570	extsh op1,op1
3571	mullw result,op1,op2
3572	SET_FLAGS(FLAGS_MUL)
3573	extsh r3,result
3574	sthbrx result,REG
3575	cmpw r3,result
3576	beq+ nop
3577	oris flags,flags,(CF_SET\|OF_SET)>>16
3578	NEXT # SF/ZF/AF/PF undefined !
3579
3580	imull_mem_reg: lwbrx op2,REG
3581	b 1f
3582
3583	imull_imm: NEXTDWORD(op2)
3584	b 1f
3585
3586	imull_imm8: NEXTBYTE(op2)
3587	extsb op2,op2
3588	1: lwbrx op1,MEM
3589	mullw result,op1,op2
3590	SET_FLAGS(FLAGS_MUL)
3591	mulhw r3,op1,op2
3592	srawi r4,result,31
3593	stwbrx result,REG
3594	cmpw r3,r4
3595	beq+ nop
3596	oris flags,flags,(CF_SET\|OF_SET)>>16
3597	NEXT # SF/ZF/AF/PF undefined !
3598
3599	/* aad is indeed a multiply */
3600	aad: NEXTBYTE(r3)
3601	lbz op1,AH(state)
3602	lbz op2,AL(state)
3603	mullw result,op1,r3 # AH*imm
3604	SET_FLAGS(FLAGS_LOG(B)) # SF/ZF/PF from result
3605	add result,result,op2 # AH*imm+AL
3606	slwi r3,result,8
3607	sth r3,AX(state) # AH=0
3608	NEXT # OF/AF/CF undefined
3609
3610	/* Unsigned divides: we may destroy all flags */
3611	divb: lhbrx r4,AX,state
3612	lbzx r3,MEM
3613	srwi r5,r4,8
3614	cmplw r5,r3
3615	bnl- _divide_error
3616	divwu r5,r4,r3
3617	mullw r3,r5,r3
3618	sub r3,r4,r3
3619	stb r5,AL(state)
3620	stb r3,AH(state)
3621	NEXT
3622
3623	divw: li opreg,DX
3624	lhbrx r4,AX,state
3625	lhbrx r5,REG
3626	lhbrx r3,MEM
3627	insrwi r4,r5,16,0
3628	cmplw r5,r3
3629	bnl- _divide_error
3630	divwu r5,r4,r3
3631	mullw r3,r5,r3
3632	sub r3,r4,r3
3633	sthbrx r5,AX,state
3634	sthbrx r3,REG
3635	NEXT
3636
3637	divl: li opreg,EDX # Not yet fully implemented
3638	lwbrx r3,MEM
3639	lwbrx r4,REG
3640	lwbrx r5,EAX,state
3641	cmplw r4,r3
3642	bnl- _divide_error
3643	cmplwi r4,0
3644	bne- 1f
3645	divwu r4,r5,r3
3646	mullw r3,r4,r3
3647	stwbrx r4,EAX,state
3648	sub r3,r5,r3
3649	stwbrx r3,REG
3650	NEXT
3651	/* full implementation of 64:32 unsigned divide, slow but rarely used */
3652	1: bl _div_64_32
3653	stwbrx r5,EAX,state
3654	stwbrx r4,REG
3655	NEXT
3656	/*
3657	* Divide r4:r5 by r3, quotient in r5, remainder in r4.
3658	* The algorithm is stupid because it won't be used very often.
3659	*/
3660	_div_64_32: li r7,32
3661	mtctr r7
3662	1: cmpwi r4,0 # always subtract in case
3663	addc r5,r5,r5 # MSB is set
3664	adde r4,r4,r4
3665	blt 2f
3666	cmplw r4,r3
3667	blt 3f
3668	2: sub r4,r4,r3
3669	addi r5,r5,1
3670	3: bdnz 1b
3671
3672	/* Signed divides: we may destroy all flags */
3673	idivb: lbzx r3,MEM
3674	lhbrx r4,AX,state
3675	cmpwi r3,0
3676	beq- _divide_error
3677	divw r5,r4,r3
3678	extsb r7,r5
3679	mullw r3,r5,r3
3680	cmpw r5,r7
3681	sub r3,r4,r3
3682	bne- _divide_error
3683	stb r5,AL(state)
3684	stb r3,AH(state)
3685	NEXT
3686
3687	idivw: li opreg,DX
3688	lhbrx r4,AX,state
3689	lhbrx r5,REG
3690	lhbrx r3,MEM
3691	insrwi r4,r5,16,0
3692	cmpwi r3,0
3693	beq- _divide_error
3694	divw r5,r4,r3
3695	extsh r7,r5
3696	mullw r3,r5,r3
3697	cmpw r5,r7
3698	sub r3,r4,r3
3699	bne- _divide_error
3700	sthbrx r5,AX,state
3701	sthbrx r3,REG
3702	NEXT
3703
3704	idivl: li opreg,EDX # Not yet fully implemented
3705	lwbrx r3,MEM
3706	lwbrx r5,EAX,state
3707	cmpwi cr1,r3,0
3708	lwbrx r4,REG
3709	srwi r7,r5,31
3710	beq- _divide_error
3711	add. r7,r7,r4
3712	bne- 1f # EDX not sign extension of EAX
3713	divw r4,r5,r3
3714	xoris r7,r5,0x8000 # only overflow case is
3715	orc. r7,r7,r3 # 0x80000000 divided by -1
3716	mullw r3,r4,r3
3717	beq- _divide_error
3718	stwbrx r4,EAX,state
3719	sub r3,r5,r3
3720	stwbrx r3,REG
3721	NEXT
3722
3723	/* full 64 by 32 signed divide, checks for overflow might be right now */
3724	1: srawi r6,r4,31 # absolute value of r4:r5
3725	srawi r0,r3,31 # absolute value of r3
3726	xor r5,r5,r6
3727	xor r3,r3,r0
3728	subfc r5,r6,r5
3729	xor r4,r4,r6
3730	sub r3,r3,r0
3731	subfe r4,r6,r4
3732	xor r0,r0,r6 # sign of result
3733	cmplw r4,r3 # coarse overflow detection
3734	bnl- _divide_error # (probably not necessary)
3735	bl _div_64_32
3736	xor r5,r5,r0 # apply sign to result
3737	sub r5,r5,r0
3738	xor. r7,r0,r5 # wrong sign: overflow
3739	xor r4,r4,r6 # apply sign to remainder
3740	blt- _divide_error
3741	stwbrx r5,EAX,state
3742	sub r4,r4,r6
3743	stwbrx r4,REG
3744	NEXT
3745
3746	/* aam is indeed a divide */
3747	aam: NEXTBYTE(r3)
3748	lbz r4,AL(state)
3749	cmpwi r3,0
3750	beq- _divide_error # zero divide
3751	divwu op2,r4,r3 # AL/imm8
3752	SET_FLAGS(FLAGS_LOG(B)) # SF/ZF/PF from AL
3753	mullw r3,op2,r3 # (AL/imm8)*imm8
3754	stb op2,AH(state)
3755	sub result,r4,r3 # AL-imm8*(AL/imm8)
3756	stb result,AL(state)
3757	NEXT # OF/AF/CF undefined
3758
3759	_divide_error: li r3,code_divide_err
3760	b complex
3761
3762	/* Instructions dealing with segment registers */
3763	pushw_sp_sr: li r3,SP
3764	rlwinm opreg,opcode,31,27,29
3765	addi r5,state,SELECTORS+2
3766	lhbrx r4,state,r3
3767	lhzx r0,r5,opreg
3768	addi r4,r4,-2
3769	sthbrx r4,state,r3
3770	clrlwi r4,r4,16
3771	sthbrx r0,r4,ssb
3772	NEXT
3773
3774	pushl_sp_sr: li r3,SP
3775	rlwinm opreg,opcode,31,27,29
3776	addi r5,state,SELECTORS+2
3777	lhbrx r4,state,r3
3778	lhzx r0,r5,opreg
3779	addi r4,r4,-4
3780	sthbrx r4,state,r3
3781	clrlwi r4,r4,16
3782	stwbrx r0,r4,ssb
3783	NEXT
3784
3785	movl_sr_mem: cmpwi opreg,20
3786	addi opreg,opreg,SELECTORS+2
3787	cmpw cr1,base,state # Only registers are sensitive
3788	bgt- ud # to word/longword difference
3789	lhzx r0,REG
3790	bne cr1,1f
3791	stwbrx r0,MEM # Actually a register
3792	NEXT
3793
3794	movw_sr_mem: cmpwi opreg,20 # SREG 0 to 5 only
3795	addi opreg,opreg,SELECTORS+2
3796	bgt- ud
3797	lhzx r0,REG
3798	1: sthbrx r0,MEM
3799	NEXT
3800
3801	/* Now the instructions that modify the segment registers, note that
3802	move/pop to ss disable interrupts and traps for one instruction ! */
3803	popl_sp_sr: li r6,4
3804	b 1f
3805	popw_sp_sr: li r6,2
3806	1: li r7,SP
3807	rlwinm opreg,opcode,31,27,29
3808	lhbrx offset,state,r7
3809	addi opreg,opreg,SELBASES
3810	lhbrx r4,ssb,offset # new selector
3811	add offset,offset,r6
3812	bl _segment_load
3813	sthbrx offset,state,r7 # update sp
3814	cmpwi opreg,8 # is ss ?
3815	stwux r3,REG
3816	stw r4,SELECTORS-SELBASES(opreg)
3817	lwz esb,esbase(state)
3818	bne+ nop
3819	lwz ssb,ssbase(state) # pop ss
3820	crmove RF,TF # prevent traps
3821	NEXT
3822
3823	movw_mem_sr: cmpwi opreg,20
3824	addi r7,state,SELBASES
3825	bgt- ud
3826	cmpwi opreg,4 # CS illegal
3827	beq- ud
3828	lhbrx r4,MEM
3829	bl _segment_load
3830	stwux r3,r7,opreg
3831	cmpwi opreg,8
3832	stw r4,SELECTORS-SELBASES(r7)
3833	lwz esb,esbase(state)
3834	bne+ nop
3835	lwz ssb,ssbase(state)
3836	crmove RF,TF # prevent traps
3837	NEXT
3838
3839	.equ movl_mem_sr, movw_mem_sr
3840
3841	/* The encoding of les/lss/lds/lfs/lgs is strange, opcode is c4/b2/c5/b4/b5
3842	for es/ss/ds/fs/gs which are sreg 0/2/3/4/5. And obviously there is
3843	no lcs instruction, it's called a far jump. */
3844
3845	ldlptrl: lwzux r7,MEM
3846	li r4,4
3847	bl 1f
3848	stwx r7,REG
3849	NEXT
3850	ldlptrw: lhzux r7,MEM
3851	li r4,2
3852	bl 1f
3853	sthx r7,REG
3854	NEXT
3855
3856	1: cmpw base,state
3857	lis r3,0xc011 # es/ss/ds/fs/gs
3858	rlwinm r5,opcode,2,0x0c # 00/08/04/00/04
3859	mflr r0
3860	addi r3,r3,0x4800 # r4=0xc0114800
3861	rlwimi r5,opcode,0,0x10 # 00/18/04/10/14
3862	lhbrx r4,r4,offset
3863	rlwnm opcode,r3,r5,0x1c # 00/08/0c/10/14 = sreg*4 !
3864	beq- ud # Only mem operands allowed !
3865	bl _segment_load
3866	addi r5,opcode,SELBASES
3867	stwux r3,r5,state
3868	mtlr r0
3869	stw r4,SELECTORS-SELBASES(r5)
3870	lwz esb,esbase(state) # keep shadow state in sync
3871	lwz ssb,ssbase(state)
3872	blr
3873
3874
3875	/* Intructions that may modify the current code segment: the next optimization
3876	* might be to avoid calling C code when the code segment does not change. But
3877	* it's probably not worth the effort.
3878	*/
3879	/* Far calls, jumps and returns */
3880	lcall_w: NEXTWORD(r4)
3881	NEXTWORD(r5)
3882	li r3,code_lcallw
3883	b complex
3884
3885	lcall_l: NEXTDWORD(r4)
3886	NEXTWORD(r5)
3887	li r3,code_lcalll
3888	b complex
3889
3890	lcallw: lhbrx r4,MEM
3891	addi offset,offset,2
3892	lhbrx r5,MEM
3893	li r3,code_lcallw
3894	b complex
3895
3896	lcalll: lwbrx r4,MEM
3897	addi offset,offset,4
3898	lhbrx r5,MEM
3899	li r3,code_lcalll
3900	b complex
3901
3902	ljmp_w: NEXTWORD(r4)
3903	NEXTWORD(r5)
3904	li r3,code_ljmpw
3905	b complex
3906
3907	ljmp_l: NEXTDWORD(r4)
3908	NEXTWORD(r5)
3909	li r3,code_ljmpl
3910	b complex
3911
3912	ljmpw: lhbrx r4,MEM
3913	addi offset,offset,2
3914	lhbrx r5,MEM
3915	li r3,code_ljmpw
3916	b complex
3917
3918	ljmpl: lwbrx r4,MEM
3919	addi offset,offset,4
3920	lhbrx r5,MEM
3921	li r3,code_ljmpl
3922	b complex
3923
3924	lretw_imm: NEXTWORD(r4)
3925	b 1f
3926	lretw: li r4,0
3927	1: li r3,code_lretw
3928	b complex
3929
3930	lretl_imm: NEXTWORD(r4)
3931	b 1f
3932	lretl: li r4,0
3933	1: li r3,code_lretl
3934	b complex
3935
3936	/* Interrupts */
3937	int: li r3,code_softint # handled by C code
3938	NEXTBYTE(r4)
3939	b complex
3940
3941	int3: li r3,code_int3 # handled by C code
3942	b complex
3943
3944	into: EVAL_OF
3945	bf+ OF,nop
3946	li r3,code_into
3947	b complex # handled by C code
3948
3949	iretw: li r3,code_iretw # handled by C code
3950	b complex
3951
3952	iretl: li r3,code_iretl
3953	b complex
3954
3955	/* Miscellaneous flag control instructions */
3956	clc: oris flags,flags,(CF_IN_CR\|CF_STATE_MASK\|ABOVE_IN_CR)>>16
3957	xoris flags,flags,(CF_IN_CR\|CF_STATE_MASK\|ABOVE_IN_CR)>>16
3958	NEXT
3959
3960	cmc: oris flags,flags,(CF_IN_CR\|ABOVE_IN_CR)>>16
3961	xoris flags,flags,(CF_IN_CR\|CF_COMPLEMENT\|ABOVE_IN_CR)>>16
3962	NEXT
3963
3964	stc: oris flags,flags,\
3965	(CF_IN_CR\|CF_LOCATION\|CF_COMPLEMENT\|ABOVE_IN_CR)>>16
3966	xoris flags,flags,(CF_IN_CR\|CF_LOCATION\|ABOVE_IN_CR)>>16
3967	NEXT
3968
3969	cld: crclr DF
3970	NEXT
3971
3972	std: crset DF
3973	NEXT
3974
3975	cli: crclr IF
3976	NEXT
3977
3978	sti: crset IF
3979	NEXT
3980
3981	lahf: bl _eval_flags
3982	stb r3,AH(state)
3983	NEXT
3984
3985	sahf: andis. r3,flags,OF_EXPLICIT>>16
3986	lbz r0,AH(state)
3987	beql+ _eval_of # save OF just in case
3988	rlwinm op1,r0,31,0x08 # AF
3989	rlwinm flags,flags,0,OF_STATE_MASK
3990	extsb result,r0 # SF/PF
3991	ZF862ZF(r0)
3992	oris flags,flags,(ZF_PROTECT\|ZF_IN_CR\|SF_IN_CR)>>16
3993	addi op2,op1,0 # AF
3994	ori result,result,0x00fb # set all except PF
3995	mtcrf 0x02,r0 # SF/ZF
3996	rlwimi flags,r0,27,CF_VALUE # CF
3997	xori result,result,0x00ff # 00 if PF set, 04 if clear
3998	NEXT
3999
4000	pushfw_sp: bl _eval_flags
4001	li r4,SP
4002	lhbrx r5,r4,state
4003	addi r5,r5,-2
4004	sthbrx r5,r4,state
4005	clrlwi r5,r5,16
4006	sthbrx r3,ssb,r5
4007	NEXT
4008
4009	pushfl_sp: bl _eval_flags
4010	li r4,SP
4011	lhbrx r5,r4,state
4012	addi r5,r5,-4
4013	sthbrx r5,r4,state
4014	clrlwi r5,r5,16
4015	stwbrx r3,ssb,r5
4016	NEXT
4017
4018	popfl_sp: li r4,SP
4019	lhbrx r5,r4,state
4020	lwbrx r3,ssb,r5
4021	addi r5,r5,4
4022	stw r3,eflags(state)
4023	sthbrx r5,r4,state
4024	b 1f
4025
4026	popfw_sp: li r4,SP
4027	lhbrx r5,r4,state
4028	lhbrx r3,ssb,r5
4029	addi r5,r5,2
4030	sth r3,eflags+2(state)
4031	sthbrx r5,r4,state
4032	1: rlwinm op1,r3,31,0x08 # AF
4033	xori result,r3,4 # PF
4034	ZF862ZF(r3) # cr6
4035	lis flags,(OF_EXPLICIT\|ZF_PROTECT\|ZF_IN_CR\|SF_IN_CR)>>16
4036	addi op2,op1,0 # AF
4037	rlwinm result,result,0,0x04 # PF
4038	rlwimi flags,r3,27,CF_VALUE # CF
4039	mtcrf 0x6,r3 # IF,DF,TF,SF,ZF
4040	rlwimi result,r3,24,0,0 # SF
4041	rlwimi flags,r3,15,OF_VALUE # OF
4042	NEXT
4043
4044	/* SETcc is slightly faster for setz/setnz */
4045	setz: EVAL_ZF
4046	bt ZF,1f
4047	0: cmpwi opreg,0
4048	bne- ud
4049	stbx opreg,MEM
4050	NEXT
4051
4052	setnz: EVAL_ZF
4053	bt ZF,0b
4054	1: cmpwi opreg,0
4055	bne- ud
4056	stbx one,MEM
4057	NEXT
4058
4059	#define SETCC(cond, eval, flag) \
4060	set##cond: EVAL_##eval; bt flag,1b; b 0b; \
4061	setn##cond: EVAL_##eval; bt flag,0b; b 1b
4062
4063	SETCC(c, CF, CF)
4064	SETCC(a, ABOVE, ABOVE)
4065	SETCC(s, SF, SF)
4066	SETCC(g, SIGNED, SGT)
4067	SETCC(l, SIGNED, SLT)
4068	SETCC(o, OF, OF)
4069	SETCC(p, PF, PF)
4070
4071	/* No wait for a 486SX */
4072	.equ wait, nop
4073
4074	/* ARPL is not recognized in real mode */
4075	.equ arpl, ud
4076
4077	/* clts and in general control and debug registers are not implemented */
4078	.equ clts, unimpl
4079
4080	aaa: lhbrx r0,AX,state
4081	bl _eval_af
4082	rlwinm r3,r3,0,0x10
4083	SET_FLAGS(FLAGS_ADD(W))
4084	rlwimi r3,r0,0,0x0f
4085	li r4,0x106
4086	addi r3,r3,-10
4087	srwi r3,r3,16 # carry ? 0 : 0xffff
4088	andc op1,r4,r3 # carry ? 0x106 : 0
4089	add result,r0,op1
4090	rlwinm result,result,0,28,23 # clear high half of AL
4091	li op2,10 # sets AF indirectly
4092	sthbrx r3,AX,state # OF/SF/ZF/PF undefined !
4093	rlwimi result,op1,8,0x10000 # insert CF
4094	NEXT
4095
4096	aas: lhbrx r0,AX,state
4097	bl _eval_af
4098	rlwinm r3,r3,0,0x10
4099	SET_FLAGS(FLAGS_ADD(W))
4100	rlwimi r3,r0,0,0x0f # AF:AL&0x0f
4101	li r4,0x106
4102	addi r3,r3,-10
4103	srwi r3,r3,16 # carry ? 0 : 0xffff
4104	andc op1,r4,r3 # carry ? 0x106 : 0
4105	sub result,r0,op1
4106	rlwinm result,result,0,28,23 # clear high half of AL
4107	li op2,10 # sets AF indirectly
4108	sthbrx r3,AX,state # OF/SF/ZF/PF undefined !
4109	rlwimi result,op1,8,0x10000 # insert CF
4110	NEXT
4111
4112	daa: lbz r0,AL(state)
4113	bl _eval_af
4114	rlwinm r7,r3,0,0x10
4115	bl _eval_cf # r3=CF<<8
4116	rlwimi r7,r0,0,0x0f
4117	SET_FLAGS(FLAGS_ADD(B))
4118	addi r4,r7,-10
4119	rlwinm r4,r4,3,0x06 # 6 if AF or >9, 0 otherwise
4120	srwi op1,r7,1 # 0..4, no AF, 5..f AF set
4121	add r0,r0,r4 # conditional add
4122	li op2,11 # sets AF depnding on op1
4123	or r0,r0,r3
4124	subfic r3,r0,159
4125	rlwinm r3,r3,7,0x60 # mask value to add
4126	add result,r0,r3 # final result for SF/ZF/PF
4127	stb result,AL(state)
4128	rlwimi result,r3,2,0x100 # set CF if added
4129	NEXT
4130
4131	das: lbz r0,AL(state)
4132	bl _eval_af
4133	rlwinm r7,r3,0,0x10
4134	bl _eval_cf
4135	rlwimi r7,r0,0,0x0f
4136	SET_FLAGS(FLAGS_ADD(B))
4137	addi r4,r7,-10
4138	rlwinm r4,r4,3,0x06
4139	srwi op1,r7,1 # 0..4, no AF, 5..f AF set
4140	sub r0,r0,r4 # conditional add
4141	li op2,11 # sets AF depending on op1
4142	or r4,r0,r3 # insert CF
4143	addi r3,r4,-160
4144	rlwinm r3,r3,7,0x60 # mask value to add
4145	sub result,r4,r3 # final result for SF/ZF/PF
4146	stb result,AL(state)
4147	rlwimi result,r3,2,0x100 # set CF
4148	NEXT
4149
4150	/* 486 specific instructions */
4151
4152	/* For cmpxchg, only the zero flag is important */
4153
4154	cmpxchgb: lbz op1,AL(state)
4155	SET_FLAGS(FLAGS_SUB(B)\|ZF_IN_CR)
4156	lbzx op2,MEM
4157	cmpw cr6,op1,op2
4158	sub result,op1,op2
4159	bne cr6,1f
4160	lbzx r3,REG # success: swap
4161	stbx r3,MEM
4162	NEXT
4163	1: stb op2,AL(state)
4164	NEXT
4165
4166	cmpxchgw: lhbrx op1,AX,state
4167	SET_FLAGS(FLAGS_SUB(W)\|ZF_IN_CR)
4168	lhbrx op2,MEM
4169	cmpw cr6,op1,op2
4170	sub result,op1,op2
4171	bne cr6,1f
4172	lhzx r3,REG # success: swap
4173	sthx r3,MEM
4174	NEXT
4175	1: sthbrx op2,AX,state
4176	NEXT
4177
4178	cmpxchgl: lwbrx op1,EAX,state
4179	SET_FLAGS(FLAGS_SUB(L)\|ZF_IN_CR\|SIGNED_IN_CR)
4180	lwbrx op2,MEM
4181	cmpw cr6,op1,op2
4182	sub result,op1,op2
4183	bne cr6,1f
4184	lwzx r3,REG # success: swap
4185	stwx r3,MEM
4186	NEXT
4187	1: stwbrx op2,EAX,state
4188	NEXT
4189
4190	xaddb: lbzx op2,MEM
4191	SET_FLAGS(FLAGS_ADD(B))
4192	lbzx op1,REG
4193	add result,op1,op2
4194	stbx result,MEM
4195	stbx op2,REG
4196	NEXT
4197
4198	xaddw: lhbrx op2,MEM
4199	SET_FLAGS(FLAGS_ADD(W))
4200	lhbrx op1,REG
4201	add result,op1,op2
4202	sthbrx result,MEM
4203	sthbrx op2,REG
4204	NEXT
4205
4206	xaddl: lwbrx op2,MEM
4207	SET_FLAGS(FLAGS_ADD(L))
4208	lwbrx op1,REG
4209	add result,op1,op2
4210	stwbrx result,MEM
4211	stwbrx op2,REG
4212	NEXT
4213
4214	/* All FPU instructions skipped. This is a 486 SX ! */
4215	esc: li r3,code_dna # DNA interrupt
4216	b complex
4217
4218	.equ hlt, unimpl # Cannot stop
4219
4220	.equ invd, unimpl
4221
4222	/* Undefined in real address mode */
4223	.equ lar, ud
4224
4225	.equ lgdt, unimpl
4226	.equ lidt, unimpl
4227	.equ lldt, ud
4228	.equ lmsw, unimpl
4229
4230	/* protected mode only */
4231	.equ lsl, ud
4232	.equ ltr, ud
4233
4234	.equ movl_cr_reg, unimpl
4235	.equ movl_reg_cr, unimpl
4236	.equ movl_dr_reg, unimpl
4237	.equ movl_reg_dr, unimpl
4238
4239	.equ sgdt, unimpl
4240
4241	.equ sidt, unimpl
4242	.equ sldt, ud
4243	.equ smsw, unimpl
4244
4245	.equ str, ud
4246
4247	ud: li r3,code_ud
4248	li r4,0
4249	b complex
4250
4251	unimpl: li r3,code_ud
4252	li r4,1
4253	b complex
4254
4255	.equ verr, ud
4256	.equ verw, ud
4257	.equ wbinvd, unimpl
4258
4259	em86_end:
4260	.size em86_enter,em86_end-em86_enter
4261	#ifdef __BOOT__
4262	.data
4263	#define ENTRY(x,t) .long x+t-_jtables
4264	#else
4265	.section .rodata
4266	#define ENTRY(x,t) .long x+t
4267	#endif
4268
4269	#define BOP(x) ENTRY(x,2) /* Byte operation with mod/rm byte */
4270	#define WLOP(x) ENTRY(x,3) /* 16 or 32 bit operation with mod/rm byte */
4271	#define EXTOP(x) ENTRY(x,0) /* Opcode with extension in mod/rm byte */
4272	#define OP(x) ENTRY(x,1) /* Direct one byte opcode/prefix */
4273
4274	/* A few macros for the main table */
4275	#define gen6(op, wl, axeax) \
4276	BOP(op##b##_reg_mem); WLOP(op##wl##_reg_mem); \
4277	BOP(op##b##_mem_reg); WLOP(op##wl##_mem_reg); \
4278	OP(op##b##_imm_al); OP(op##wl##_imm_##axeax)
4279
4280	#define rep7(l,t) \
4281	ENTRY(l,t); ENTRY(l,t); ENTRY(l,t); ENTRY(l,t); \
4282	ENTRY(l,t); ENTRY(l,t); ENTRY(l,t)
4283
4284	#define rep8(l) l ; l; l; l; l; l; l; l;
4285
4286	#define allcond(pfx, sfx, t) \
4287	ENTRY(pfx##o##sfx, t); ENTRY(pfx##no##sfx, t); \
4288	ENTRY(pfx##c##sfx, t); ENTRY(pfx##nc##sfx, t); \
4289	ENTRY(pfx##z##sfx, t); ENTRY(pfx##nz##sfx, t); \
4290	ENTRY(pfx##na##sfx, t); ENTRY(pfx##a##sfx, t); \
4291	ENTRY(pfx##s##sfx, t); ENTRY(pfx##ns##sfx, t); \
4292	ENTRY(pfx##p##sfx, t); ENTRY(pfx##np##sfx, t); \
4293	ENTRY(pfx##l##sfx, t); ENTRY(pfx##nl##sfx, t); \
4294	ENTRY(pfx##ng##sfx, t); ENTRY(pfx##g##sfx, t)
4295
4296	/* single/double register sign extensions and other oddities */
4297	#define h2sextw cbw /* Half to Single sign extension */
4298	#define s2dextw cwd /* Single to Double sign extension */
4299	#define h2sextl cwde
4300	#define s2dextl cdq
4301	#define j_a16_cxz_w jcxz_w
4302	#define j_a32_cxz_w jecxz_w
4303	#define j_a16_cxz_l jcxz_l
4304	#define j_a32_cxz_l jecxz_l
4305	#define loopa16_w loopw_w
4306	#define loopa16_l loopw_l
4307	#define loopa32_w loopl_w
4308	#define loopa32_l loopl_l
4309	#define loopnza16_w loopnzw_w
4310	#define loopnza16_l loopnzw_l
4311	#define loopnza32_w loopnzl_w
4312	#define loopnza32_l loopnzl_l
4313	#define loopza16_w loopzw_w
4314	#define loopza16_l loopzw_l
4315	#define loopza32_w loopzl_w
4316	#define loopza32_l loopzl_l
4317	/* No FP support */
4318
4319	/* Addressing mode table */
4320	.align 5
4321	# (%bx,%si), (%bx,%di), (%bp,%si), (%bp,%di)
4322	adtable: .long 0x00004360, 0x00004370, 0x80004560, 0x80004570
4323	# (%si), (%di), o16, (%bx)
4324	.long 0x00004600, 0x00004700, 0x00002000, 0x00004300
4325	# o8(%bx,%si), o8(%bx,%di), o8(%bp,%si), o8(%bp,%di)
4326	.long 0x00004360, 0x00004370, 0x80004560, 0x80004570
4327	# o8(%si), o8(%di), o8(%bp), o8(%bx)
4328	.long 0x00004600, 0x00004700, 0x80004500, 0x00004300
4329	# o16(%bx,%si), o16(%bx,%di), o16(%bp,%si), o16(%bp,%di)
4330	.long 0x00004360, 0x00004370, 0x80004560, 0x80004570
4331	# o16(%si), o16(%di), o16(%bp), o16(%bx)
4332	.long 0x00004600, 0x00004700, 0x80004500, 0x00004300
4333	# register addressing modes do not use the table
4334	.long 0, 0, 0, 0, 0, 0, 0, 0
4335	#now 32 bit modes
4336	# (%eax), (%ecx), (%edx), (%ebx)
4337	.long 0x00004090, 0x00004190, 0x00004290, 0x00004390
4338	# sib, o32, (%esi), (%edi)
4339	.long 0x00003090, 0x00002090, 0x00004690, 0x00004790
4340	# o8(%eax), o8(%ecx), o8(%edx), o8(%ebx)
4341	.long 0x00004090, 0x00004190, 0x00004290, 0x00004390
4342	# sib, o8(%ebp), o8(%esi), o8(%edi)
4343	.long 0x00003090, 0x80004590, 0x00004690, 0x00004790
4344	# o32(%eax), o32(%ecx), o32(%edx), o32(%ebx)
4345	.long 0x00004090, 0x00004190, 0x00004290, 0x00004390
4346	# sib, o32(%ebp), o32(%esi), o32(%edi)
4347	.long 0x00003090, 0x80004590, 0x00004690, 0x00004790
4348	# register addressing modes do not use the table
4349	.long 0, 0, 0, 0, 0, 0, 0, 0
4350
4351	#define jtable(wl, awl, spesp, axeax, name ) \
4352	.align 5; \
4353	jtab_##name: gen6(add, wl, axeax); \
4354	OP(push##wl##_##spesp##_sr); \
4355	OP(pop##wl##_##spesp##_sr); \
4356	gen6(or, wl, axeax); \
4357	OP(push##wl##_##spesp##_sr); \
4358	OP(_twobytes); \
4359	gen6(adc, wl, axeax); \
4360	OP(push##wl##_##spesp##_sr); \
4361	OP(pop##wl##_##spesp##_sr); \
4362	gen6(sbb, wl, axeax); \
4363	OP(push##wl##_##spesp##_sr); \
4364	OP(pop##wl##_##spesp##_sr); \
4365	gen6(and, wl, axeax); OP(_es); OP(daa); \
4366	gen6(sub, wl, axeax); OP(_cs); OP(das); \
4367	gen6(xor, wl, axeax); OP(_ss); OP(aaa); \
4368	gen6(cmp, wl, axeax); OP(_ds); OP(aas); \
4369	rep8(OP(inc##wl##_reg)); \
4370	rep8(OP(dec##wl##_reg)); \
4371	rep8(OP(push##wl##_##spesp##_reg)); \
4372	rep8(OP(pop##wl##_##spesp##_reg)); \
4373	OP(pusha##wl##_##spesp); OP(popa##wl##_##spesp); \
4374	WLOP(bound##wl); WLOP(arpl); \
4375	OP(_fs); OP(_gs); OP(_opsize); OP(_adsize); \
4376	OP(push##wl##_##spesp##_imm); WLOP(imul##wl##_imm); \
4377	OP(push##wl##_##spesp##_imm8); WLOP(imul##wl##_imm8); \
4378	OP(insb_##awl); OP(ins##wl##_##awl); \
4379	OP(outsb_##awl); OP(outs##wl##_##awl); \
4380	allcond(sj,_##wl,1); \
4381	EXTOP(grp1b_imm); EXTOP(grp1##wl##_imm); \
4382	EXTOP(grp1b_imm); EXTOP(grp1##wl##_imm8); \
4383	BOP(testb_reg_mem); WLOP(test##wl##_reg_mem); \
4384	BOP(xchgb_reg_mem); WLOP(xchg##wl##_reg_mem); \
4385	BOP(movb_reg_mem); WLOP(mov##wl##_reg_mem); \
4386	BOP(movb_mem_reg); WLOP(mov##wl##_mem_reg); \
4387	WLOP(mov##wl##_sr_mem); WLOP(lea##wl); \
4388	WLOP(mov##wl##_mem_sr); WLOP(pop##wl##_##spesp##_##awl); \
4389	OP(nop); rep7(xchg##wl##_##axeax##_reg,1); \
4390	OP(h2sext##wl); OP(s2dext##wl); \
4391	OP(lcall_##wl); OP(wait); \
4392	OP(pushf##wl##_##spesp); OP(popf##wl##_##spesp); \
4393	OP(sahf); OP(lahf); \
4394	OP(movb_##awl##_al); OP(mov##wl##_##awl##_##axeax); \
4395	OP(movb_al_##awl); OP(mov##wl##_##axeax##_##awl); \
4396	OP(movsb_##awl); OP(movs##wl##_##awl); \
4397	OP(cmpsb_##awl); OP(cmps##wl##_##awl); \
4398	OP(testb_imm_al); OP(test##wl##_imm_##axeax); \
4399	OP(stosb_##awl); OP(stos##wl##_##awl); \
4400	OP(lodsb_##awl); OP(lods##wl##_##awl); \
4401	OP(scasb_##awl); OP(scas##wl##_##awl); \
4402	rep8(OP(movb_imm_reg)); \
4403	rep8(OP(mov##wl##_imm_reg)); \
4404	EXTOP(shiftb_imm); EXTOP(shift##wl##_imm); \
4405	OP(ret##wl##_##spesp##_imm); OP(ret##wl##_##spesp); \
4406	WLOP(ldlptr##wl); WLOP(ldlptr##wl); \
4407	BOP(movb_imm_mem); WLOP(mov##wl##_imm_mem); \
4408	OP(enter##wl##_##spesp); OP(leave##wl##_##spesp); \
4409	OP(lret##wl##_imm); OP(lret##wl); \
4410	OP(int3); OP(int); OP(into); OP(iret##wl); \
4411	EXTOP(shiftb_1); EXTOP(shift##wl##_1); \
4412	EXTOP(shiftb_cl); EXTOP(shift##wl##_cl); \
4413	OP(aam); OP(aad); OP(ud); OP(xlatb_##awl); \
4414	rep8(OP(esc)); \
4415	OP(loopnz##awl##_##wl); OP(loopz##awl##_##wl); \
4416	OP(loop##awl##_##wl); OP(j_##awl##_cxz_##wl); \
4417	OP(inb_port_al); OP(in##wl##_port_##axeax); \
4418	OP(outb_al_port); OP(out##wl##_##axeax##_port); \
4419	OP(call##wl##_##spesp); OP(jmp_##wl); \
4420	OP(ljmp_##wl); OP(sjmp_##wl); \
4421	OP(inb_dx_al); OP(in##wl##_dx_##axeax); \
4422	OP(outb_al_dx); OP(out##wl##_##axeax##_dx); \
4423	OP(_lock); OP(ud); OP(_repnz); OP(_repz); \
4424	OP(hlt); OP(cmc); \
4425	EXTOP(grp3b); EXTOP(grp3##wl); \
4426	OP(clc); OP(stc); OP(cli); OP(sti); \
4427	OP(cld); OP(std); \
4428	EXTOP(grp4b); EXTOP(grp5##wl##_##spesp); \
4429	/* Here we start the table for twobyte instructions */ \
4430	OP(ud); OP(ud); WLOP(lar); WLOP(lsl); \
4431	OP(ud); OP(ud); OP(clts); OP(ud); \
4432	OP(invd); OP(wbinvd); OP(ud); OP(ud); \
4433	OP(ud); OP(ud); OP(ud); OP(ud); \
4434	rep8(OP(ud)); \
4435	rep8(OP(ud)); \
4436	OP(movl_cr_reg); OP(movl_reg_cr); \
4437	OP(movl_dr_reg); OP(movl_reg_dr); \
4438	OP(ud); OP(ud); OP(ud); OP(ud); \
4439	rep8(OP(ud)); \
4440	/* .long wrmsr, rdtsc, rdmsr, rdpmc; */\
4441	rep8(OP(ud)); \
4442	rep8(OP(ud)); \
4443	/* allcond(cmov, wl); */ \
4444	rep8(OP(ud)); rep8(OP(ud)); \
4445	rep8(OP(ud)); rep8(OP(ud)); \
4446	/* MMX Start */ \
4447	rep8(OP(ud)); rep8(OP(ud)); \
4448	rep8(OP(ud)); rep8(OP(ud)); \
4449	/* MMX End */ \
4450	allcond(j,_##wl, 1); \
4451	allcond(set,,2); \
4452	OP(push##wl##_##spesp##_sr); OP(pop##wl##_##spesp##_sr); \
4453	OP(ud) /* cpuid */; WLOP(bt##wl##_reg_mem); \
4454	WLOP(shld##wl##_imm); WLOP(shld##wl##_cl); \
4455	OP(ud); OP(ud); \
4456	OP(push##wl##_##spesp##_sr); OP(pop##wl##_##spesp##_sr); \
4457	OP(ud) /* rsm */; WLOP(bts##wl##_reg_mem); \
4458	WLOP(shrd##wl##_imm); WLOP(shrd##wl##_cl); \
4459	OP(ud); WLOP(imul##wl##_mem_reg); \
4460	BOP(cmpxchgb); WLOP(cmpxchg##wl); \
4461	WLOP(ldlptr##wl); WLOP(btr##wl##_reg_mem); \
4462	WLOP(ldlptr##wl); WLOP(ldlptr##wl); \
4463	WLOP(movzb##wl); WLOP(movzw##wl); \
4464	OP(ud); OP(ud); \
4465	EXTOP(grp8##wl); WLOP(btc##wl##_reg_mem); \
4466	WLOP(bsf##wl); WLOP(bsr##wl); \
4467	WLOP(movsb##wl); WLOP(movsw##wl); \
4468	BOP(xaddb); WLOP(xadd##wl); \
4469	OP(ud); OP(ud); \
4470	OP(ud); OP(ud); OP(ud); OP(ud); \
4471	rep8(OP(bswap)); \
4472	/* MMX Start */ \
4473	rep8(OP(ud)); rep8(OP(ud)); \
4474	rep8(OP(ud)); rep8(OP(ud)); \
4475	rep8(OP(ud)); rep8(OP(ud)); \
4476	/* MMX End */
4477	.align 5 /* 8kb of tables, 32 byte aligned */
4478	_jtables: jtable(w, a16, sp, ax, www) /* data16, addr16 */
4479	jtable(l, a16, sp, eax, lww) /* data32, addr16 */
4480	jtable(w, a32, sp, ax, wlw) /* data16, addr32 */
4481	jtable(l, a32, sp, eax, llw) /* data32, addr32 */
4482	/* The other possible combinations are only required by protected mode
4483	code using a big stack segment */
4484	/* Here are the auxiliary tables for opcode extensions, note that
4485	all entries get 2 or 3 added. */
4486	#define grp1table(bwl,t,s8) \
4487	grp1##bwl##_imm##s8:; \
4488	ENTRY(add##bwl##_imm##s8,t); ENTRY(or##bwl##_imm##s8,t); \
4489	ENTRY(adc##bwl##_imm##s8,t); ENTRY(sbb##bwl##_imm##s8,t); \
4490	ENTRY(and##bwl##_imm##s8,t); ENTRY(sub##bwl##_imm##s8,t); \
4491	ENTRY(xor##bwl##_imm##s8,t); ENTRY(cmp##bwl##_imm##s8,t)
4492
4493	grp1table(b,2,)
4494	grp1table(w,3,)
4495	grp1table(w,3,8)
4496	grp1table(l,3,)
4497	grp1table(l,3,8)
4498
4499	#define shifttable(bwl,t,c) \
4500	shift##bwl##_##c:; \
4501	ENTRY(rol##bwl##_##c,t); ENTRY(ror##bwl##_##c,t); \
4502	ENTRY(rcl##bwl##_##c,t); ENTRY(rcr##bwl##_##c,t); \
4503	ENTRY(shl##bwl##_##c,t); ENTRY(shr##bwl##_##c,t); \
4504	OP(ud); ENTRY(sar##bwl##_##c,t)
4505
4506	shifttable(b,2,1)
4507	shifttable(w,3,1)
4508	shifttable(l,3,1)
4509
4510	shifttable(b,2,cl)
4511	shifttable(w,3,cl)
4512	shifttable(l,3,cl)
4513
4514	shifttable(b,2,imm)
4515	shifttable(w,3,imm)
4516	shifttable(l,3,imm)
4517
4518	#define grp3table(bwl,t) \
4519	grp3##bwl: ENTRY(test##bwl##_imm,t); OP(ud); \
4520	ENTRY(not##bwl,t); ENTRY(neg##bwl,t); \
4521	ENTRY(mul##bwl,t); ENTRY(imul##bwl,t); \
4522	ENTRY(div##bwl,t); ENTRY(idiv##bwl,t)
4523
4524	grp3table(b,2)
4525	grp3table(w,3)
4526	grp3table(l,3)
4527
4528
4529	grp4b: BOP(incb); BOP(decb); \
4530	OP(ud); OP(ud); \
4531	OP(ud); OP(ud); \
4532	OP(ud); OP(ud)
4533
4534	#define grp5table(wl,spesp) \
4535	grp5##wl##_##spesp: \
4536	WLOP(inc##wl); WLOP(dec##wl); \
4537	WLOP(call##wl##_##spesp##_mem); WLOP(lcall##wl##); \
4538	WLOP(jmp##wl); WLOP(ljmp##wl); \
4539	WLOP(push##wl##_##spesp); OP(ud)
4540
4541	grp5table(w,sp)
4542	grp5table(l,sp)
4543
4544	#define grp8table(wl) \
4545	grp8##wl: OP(ud); OP(ud); OP(ud); OP(ud); \
4546	WLOP(bt##wl##_imm); WLOP(bts##wl##_imm); \
4547	WLOP(btr##wl##_imm); WLOP(btc##wl##_imm)
4548
4549	grp8table(w)
4550	grp8table(l)
4551	#ifdef __BOOT__
4552	_endjtables: .long 0 /* Points to _jtables after relocation */
4553	#endif
4554

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format