Mirrors/Ryujinx
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add a new JIT compiler for CPU code (#693)

Browse source 
* Start of the ARMeilleure project

* Refactoring around the old IRAdapter, now renamed to PreAllocator

* Optimize the LowestBitSet method

* Add CLZ support and fix CLS implementation

* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks

* Implement the ByteSwap IR instruction, and some refactoring on the assembler

* Implement the DivideUI IR instruction and fix 64-bits IDIV

* Correct constant operand type on CSINC

* Move division instructions implementation to InstEmitDiv

* Fix destination type for the ConditionalSelect IR instruction

* Implement UMULH and SMULH, with new IR instructions

* Fix some issues with shift instructions

* Fix constant types for BFM instructions

* Fix up new tests using the new V128 struct

* Update tests

* Move DIV tests to a separate file

* Add support for calls, and some instructions that depends on them

* Start adding support for SIMD & FP types, along with some of the related ARM instructions

* Fix some typos and the divide instruction with FP operands

* Fix wrong method call on Clz_V

* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes

* Implement SIMD logical instructions and more misc. fixes

* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations

* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes

* Implement SIMD shift instruction and fix Dup_V

* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table

* Fix check with tolerance on tester

* Implement FP & SIMD comparison instructions, and some fixes

* Update FCVT (Scalar) encoding on the table to support the Half-float variants

* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes

* Use old memory access methods, made a start on SIMD memory insts support, some fixes

* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes

* Fix arguments count with struct return values, other fixes

* More instructions

* Misc. fixes and integrate LDj3SNuD fixes

* Update tests

* Add a faster linear scan allocator, unwinding support on windows, and other changes

* Update Ryujinx.HLE

* Update Ryujinx.Graphics

* Fix V128 return pointer passing, RCX is clobbered

* Update Ryujinx.Tests

* Update ITimeZoneService

* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks

* Use generic GetFunctionPointerForDelegate method and other tweaks

* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics

* Remove some unused code on the assembler

* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler

* Add hardware capability detection

* Fix regression on Sha1h and revert Fcm** changes

* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator

* Fix silly mistake introduced on last commit on CpuId

* Generate inline stack probes when the stack allocation is too large

* Initial support for the System-V ABI

* Support multiple destination operands

* Fix SSE2 VectorInsert8 path, and other fixes

* Change placement of XMM callee save and restore code to match other compilers

* Rename Dest to Destination and Inst to Instruction

* Fix a regression related to calls and the V128 type

* Add an extra space on comments to match code style

* Some refactoring

* Fix vector insert FP32 SSE2 path

* Port over the ARM32 instructions

* Avoid memory protection races on JIT Cache

* Another fix on VectorInsert FP32 (thanks to LDj3SNuD

* Float operands don't need to use the same register when VEX is supported

* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks

* Some nits, small improvements on the pre allocator

* CpuThreadState is gone

* Allow changing CPU emulators with a config entry

* Add runtime identifiers on the ARMeilleure project

* Allow switching between CPUs through a config entry (pt. 2)

* Change win10-x64 to win-x64 on projects

* Update the Ryujinx project to use ARMeilleure

* Ensure that the selected register is valid on the hybrid allocator

* Allow exiting on returns to 0 (should fix test regression)

* Remove register assignments for most used variables on the hybrid allocator

* Do not use fixed registers as spill temp

* Add missing namespace and remove unneeded using

* Address PR feedback

* Fix types, etc

* Enable AssumeStrictAbiCompliance by default

* Ensure that Spill and Fill don't load or store any more than necessary
This commit is contained in:
gdkchan 2019-08-08 15:56:22 -03:00 • committed by emmauss
parent 1ba58e9942
commit a731ab3a2a
310 changed files with 37389 additions and 2086 deletions
Download patch file Download diff file Expand all files Collapse all files

83
ARMeilleure/IntermediateRepresentation/BasicBlock.cs Normal file
View file

@ -0,0 +1,83 @@
using System.Collections.Generic;
namespace ARMeilleure.IntermediateRepresentation
{
class BasicBlock
{
public int Index { get; set; }
public LinkedListNode<BasicBlock> Node { get; set; }
public LinkedList<Node> Operations { get; }
private BasicBlock _next;
private BasicBlock _branch;
public BasicBlock Next
{
get => _next;
set => _next = AddSuccessor(_next, value);
}
public BasicBlock Branch
{
get => _branch;
set => _branch = AddSuccessor(_branch, value);
}
public List<BasicBlock> Predecessors { get; }
public HashSet<BasicBlock> DominanceFrontiers { get; }
public BasicBlock ImmediateDominator { get; set; }
public BasicBlock()
{
Operations = new LinkedList<Node>();
Predecessors = new List<BasicBlock>();
DominanceFrontiers = new HashSet<BasicBlock>();
Index = -1;
}
public BasicBlock(int index) : this()
{
Index = index;
}
private BasicBlock AddSuccessor(BasicBlock oldBlock, BasicBlock newBlock)
{
oldBlock?.Predecessors.Remove(this);
newBlock?.Predecessors.Add(this);
return newBlock;
}
public void Append(Node node)
{
// If the branch block is not null, then the list of operations
// should end with a branch instruction. We insert the new operation
// before this branch.
if (_branch != null || (Operations.Last != null && IsLeafBlock()))
{
Operations.AddBefore(Operations.Last, node);
}
else
{
Operations.AddLast(node);
}
}
private bool IsLeafBlock()
{
return _branch == null && _next == null;
}
public Node GetLastOp()
{
return Operations.Last?.Value;
}
}
}

79
ARMeilleure/IntermediateRepresentation/Instruction.cs Normal file
View file

@ -0,0 +1,79 @@
namespace ARMeilleure.IntermediateRepresentation
{
enum Instruction
{
Add,
BitwiseAnd,
BitwiseExclusiveOr,
BitwiseNot,
BitwiseOr,
Branch,
BranchIfFalse,
BranchIfTrue,
ByteSwap,
Call,
CompareAndSwap128,
CompareEqual,
CompareGreater,
CompareGreaterOrEqual,
CompareGreaterOrEqualUI,
CompareGreaterUI,
CompareLess,
CompareLessOrEqual,
CompareLessOrEqualUI,
CompareLessUI,
CompareNotEqual,
ConditionalSelect,
ConvertI64ToI32,
ConvertToFP,
ConvertToFPUI,
Copy,
CountLeadingZeros,
Divide,
DivideUI,
Load,
Load16,
Load8,
LoadArgument,
Multiply,
Multiply64HighSI,
Multiply64HighUI,
Negate,
Return,
RotateRight,
ShiftLeft,
ShiftRightSI,
ShiftRightUI,
SignExtend16,
SignExtend32,
SignExtend8,
StackAlloc,
Store,
Store16,
Store8,
Subtract,
VectorCreateScalar,
VectorExtract,
VectorExtract16,
VectorExtract8,
VectorInsert,
VectorInsert16,
VectorInsert8,
VectorOne,
VectorZero,
VectorZeroUpper64,
VectorZeroUpper96,
ZeroExtend16,
ZeroExtend32,
ZeroExtend8,
Clobber,
CpuId,
Extended,
Fill,
LoadFromContext,
Spill,
SpillArg,
StoreToContext
}
}

138
ARMeilleure/IntermediateRepresentation/Intrinsic.cs Normal file
View file

@ -0,0 +1,138 @@
namespace ARMeilleure.IntermediateRepresentation
{
enum Intrinsic
{
X86Addpd,
X86Addps,
X86Addsd,
X86Addss,
X86Andnpd,
X86Andnps,
X86Cmppd,
X86Cmpps,
X86Cmpsd,
X86Cmpss,
X86Comisdeq,
X86Comisdge,
X86Comisdlt,
X86Comisseq,
X86Comissge,
X86Comisslt,
X86Cvtdq2pd,
X86Cvtdq2ps,
X86Cvtpd2dq,
X86Cvtpd2ps,
X86Cvtps2dq,
X86Cvtps2pd,
X86Cvtsd2si,
X86Cvtsd2ss,
X86Cvtss2sd,
X86Divpd,
X86Divps,
X86Divsd,
X86Divss,
X86Haddpd,
X86Haddps,
X86Maxpd,
X86Maxps,
X86Maxsd,
X86Maxss,
X86Minpd,
X86Minps,
X86Minsd,
X86Minss,
X86Movhlps,
X86Movlhps,
X86Mulpd,
X86Mulps,
X86Mulsd,
X86Mulss,
X86Paddb,
X86Paddd,
X86Paddq,
X86Paddw,
X86Pand,
X86Pandn,
X86Pavgb,
X86Pavgw,
X86Pblendvb,
X86Pcmpeqb,
X86Pcmpeqd,
X86Pcmpeqq,
X86Pcmpeqw,
X86Pcmpgtb,
X86Pcmpgtd,
X86Pcmpgtq,
X86Pcmpgtw,
X86Pmaxsb,
X86Pmaxsd,
X86Pmaxsw,
X86Pmaxub,
X86Pmaxud,
X86Pmaxuw,
X86Pminsb,
X86Pminsd,
X86Pminsw,
X86Pminub,
X86Pminud,
X86Pminuw,
X86Pmovsxbw,
X86Pmovsxdq,
X86Pmovsxwd,
X86Pmovzxbw,
X86Pmovzxdq,
X86Pmovzxwd,
X86Pmulld,
X86Pmullw,
X86Popcnt,
X86Por,
X86Pshufb,
X86Pslld,
X86Pslldq,
X86Psllq,
X86Psllw,
X86Psrad,
X86Psraw,
X86Psrld,
X86Psrlq,
X86Psrldq,
X86Psrlw,
X86Psubb,
X86Psubd,
X86Psubq,
X86Psubw,
X86Punpckhbw,
X86Punpckhdq,
X86Punpckhqdq,
X86Punpckhwd,
X86Punpcklbw,
X86Punpckldq,
X86Punpcklqdq,
X86Punpcklwd,
X86Pxor,
X86Rcpps,
X86Rcpss,
X86Roundpd,
X86Roundps,
X86Roundsd,
X86Roundss,
X86Rsqrtps,
X86Rsqrtss,
X86Shufpd,
X86Shufps,
X86Sqrtpd,
X86Sqrtps,
X86Sqrtsd,
X86Sqrtss,
X86Subpd,
X86Subps,
X86Subsd,
X86Subss,
X86Unpckhpd,
X86Unpckhps,
X86Unpcklpd,
X86Unpcklps,
X86Xorpd,
X86Xorps
}
}

12
ARMeilleure/IntermediateRepresentation/IntrinsicOperation.cs Normal file
View file

@ -0,0 +1,12 @@
namespace ARMeilleure.IntermediateRepresentation
{
class IntrinsicOperation : Operation
{
public Intrinsic Intrinsic { get; }
public IntrinsicOperation(Intrinsic intrin, Operand dest, params Operand[] sources) : base(Instruction.Extended, dest, sources)
{
Intrinsic = intrin;
}
}
}

25
ARMeilleure/IntermediateRepresentation/MemoryOperand.cs Normal file
View file

@ -0,0 +1,25 @@
namespace ARMeilleure.IntermediateRepresentation
{
class MemoryOperand : Operand
{
public Operand BaseAddress { get; set; }
public Operand Index { get; set; }
public Multiplier Scale { get; }
public int Displacement { get; }
public MemoryOperand(
OperandType type,
Operand baseAddress,
Operand index = null,
Multiplier scale = Multiplier.x1,
int displacement = 0) : base(OperandKind.Memory, type)
{
BaseAddress = baseAddress;
Index = index;
Scale = scale;
Displacement = displacement;
}
}
}

10
ARMeilleure/IntermediateRepresentation/Multiplier.cs Normal file
View file

@ -0,0 +1,10 @@
namespace ARMeilleure.IntermediateRepresentation
{
enum Multiplier
{
x1 = 0,
x2 = 1,
x4 = 2,
x8 = 3
}
}

163
ARMeilleure/IntermediateRepresentation/Node.cs Normal file
View file

@ -0,0 +1,163 @@
using System;
using System.Collections.Generic;
namespace ARMeilleure.IntermediateRepresentation
{
class Node
{
public Operand Destination
{
get
{
return _destinations.Length != 0 ? GetDestination(0) : null;
}
set
{
if (value != null)
{
SetDestinations(new Operand[] { value });
}
else
{
SetDestinations(new Operand[0]);
}
}
}
private Operand[] _destinations;
private Operand[] _sources;
private LinkedListNode<Node>[] _asgUseNodes;
private LinkedListNode<Node>[] _srcUseNodes;
public int DestinationsCount => _destinations.Length;
public int SourcesCount => _sources.Length;
public Node(Operand destination, int sourcesCount)
{
Destination = destination;
_sources = new Operand[sourcesCount];
_srcUseNodes = new LinkedListNode<Node>[sourcesCount];
}
public Node(Operand[] destinations, int sourcesCount)
{
SetDestinations(destinations ?? throw new ArgumentNullException(nameof(destinations)));
_sources = new Operand[sourcesCount];
_srcUseNodes = new LinkedListNode<Node>[sourcesCount];
}
public Operand GetDestination(int index)
{
return _destinations[index];
}
public Operand GetSource(int index)
{
return _sources[index];
}
public void SetDestination(int index, Operand destination)
{
Operand oldOp = _destinations[index];
if (oldOp != null && oldOp.Kind == OperandKind.LocalVariable)
{
oldOp.Assignments.Remove(_asgUseNodes[index]);
}
if (destination != null && destination.Kind == OperandKind.LocalVariable)
{
_asgUseNodes[index] = destination.Assignments.AddLast(this);
}
_destinations[index] = destination;
}
public void SetSource(int index, Operand source)
{
Operand oldOp = _sources[index];
if (oldOp != null && oldOp.Kind == OperandKind.LocalVariable)
{
oldOp.Uses.Remove(_srcUseNodes[index]);
}
if (source != null && source.Kind == OperandKind.LocalVariable)
{
_srcUseNodes[index] = source.Uses.AddLast(this);
}
_sources[index] = source;
}
public void SetDestinations(Operand[] destinations)
{
if (_destinations != null)
{
for (int index = 0; index < _destinations.Length; index++)
{
Operand oldOp = _destinations[index];
if (oldOp != null && oldOp.Kind == OperandKind.LocalVariable)
{
oldOp.Assignments.Remove(_asgUseNodes[index]);
}
}
_destinations = destinations;
}
else
{
_destinations = new Operand[destinations.Length];
}
_asgUseNodes = new LinkedListNode<Node>[destinations.Length];
for (int index = 0; index < destinations.Length; index++)
{
Operand newOp = destinations[index];
_destinations[index] = newOp;
if (newOp.Kind == OperandKind.LocalVariable)
{
_asgUseNodes[index] = newOp.Assignments.AddLast(this);
}
}
}
public void SetSources(Operand[] sources)
{
for (int index = 0; index < _sources.Length; index++)
{
Operand oldOp = _sources[index];
if (oldOp != null && oldOp.Kind == OperandKind.LocalVariable)
{
oldOp.Uses.Remove(_srcUseNodes[index]);
}
}
_sources = new Operand[sources.Length];
_srcUseNodes = new LinkedListNode<Node>[sources.Length];
for (int index = 0; index < sources.Length; index++)
{
Operand newOp = sources[index];
_sources[index] = newOp;
if (newOp.Kind == OperandKind.LocalVariable)
{
_srcUseNodes[index] = newOp.Uses.AddLast(this);
}
}
}
}
}

124
ARMeilleure/IntermediateRepresentation/Operand.cs Normal file
View file

@ -0,0 +1,124 @@
using System;
using System.Collections.Generic;
namespace ARMeilleure.IntermediateRepresentation
{
class Operand
{
public OperandKind Kind { get; }
public OperandType Type { get; }
public ulong Value { get; private set; }
public LinkedList<Node> Assignments { get; }
public LinkedList<Node> Uses { get; }
private Operand()
{
Assignments = new LinkedList<Node>();
Uses = new LinkedList<Node>();
}
public Operand(OperandKind kind, OperandType type = OperandType.None) : this()
{
Kind = kind;
Type = type;
}
public Operand(int value) : this(OperandKind.Constant, OperandType.I32)
{
Value = (uint)value;
}
public Operand(uint value) : this(OperandKind.Constant, OperandType.I32)
{
Value = (uint)value;
}
public Operand(long value) : this(OperandKind.Constant, OperandType.I64)
{
Value = (ulong)value;
}
public Operand(ulong value) : this(OperandKind.Constant, OperandType.I64)
{
Value = value;
}
public Operand(float value) : this(OperandKind.Constant, OperandType.FP32)
{
Value = (ulong)BitConverter.SingleToInt32Bits(value);
}
public Operand(double value) : this(OperandKind.Constant, OperandType.FP64)
{
Value = (ulong)BitConverter.DoubleToInt64Bits(value);
}
public Operand(int index, RegisterType regType, OperandType type) : this()
{
Kind = OperandKind.Register;
Type = type;
Value = (ulong)((int)regType << 24 | index);
}
public Register GetRegister()
{
return new Register((int)Value & 0xffffff, (RegisterType)(Value >> 24));
}
public byte AsByte()
{
return (byte)Value;
}
public short AsInt16()
{
return (short)Value;
}
public int AsInt32()
{
return (int)Value;
}
public long AsInt64()
{
return (long)Value;
}
public float AsFloat()
{
return BitConverter.Int32BitsToSingle((int)Value);
}
public double AsDouble()
{
return BitConverter.Int64BitsToDouble((long)Value);
}
internal void NumberLocal(int number)
{
if (Kind != OperandKind.LocalVariable)
{
throw new InvalidOperationException("The operand is not a local variable.");
}
Value = (ulong)number;
}
public override int GetHashCode()
{
if (Kind == OperandKind.LocalVariable)
{
return base.GetHashCode();
}
else
{
return (int)Value ^ ((int)Kind << 16) ^ ((int)Type << 20);
}
}
}
}

68
ARMeilleure/IntermediateRepresentation/OperandHelper.cs Normal file
View file

@ -0,0 +1,68 @@
using ARMeilleure.State;
using System;
namespace ARMeilleure.IntermediateRepresentation
{
static class OperandHelper
{
public static Operand Const(OperandType type, long value)
{
return type == OperandType.I32 ? new Operand((int)value) : new Operand(value);
}
public static Operand Const(bool value)
{
return new Operand(value ? 1 : 0);
}
public static Operand Const(int value)
{
return new Operand(value);
}
public static Operand Const(uint value)
{
return new Operand(value);
}
public static Operand Const(long value)
{
return new Operand(value);
}
public static Operand Const(ulong value)
{
return new Operand(value);
}
public static Operand ConstF(float value)
{
return new Operand(value);
}
public static Operand ConstF(double value)
{
return new Operand(value);
}
public static Operand Label()
{
return new Operand(OperandKind.Label);
}
public static Operand Local(OperandType type)
{
return new Operand(OperandKind.LocalVariable, type);
}
public static Operand Register(int index, RegisterType regType, OperandType type)
{
return new Operand(index, regType, type);
}
public static Operand Undef()
{
return new Operand(OperandKind.Undefined);
}
}
}

12
ARMeilleure/IntermediateRepresentation/OperandKind.cs Normal file
View file

@ -0,0 +1,12 @@
namespace ARMeilleure.IntermediateRepresentation
{
enum OperandKind
{
Constant,
Label,
LocalVariable,
Memory,
Register,
Undefined
}
}

51
ARMeilleure/IntermediateRepresentation/OperandType.cs Normal file
View file

@ -0,0 +1,51 @@
using System;
namespace ARMeilleure.IntermediateRepresentation
{
enum OperandType
{
None,
I32,
I64,
FP32,
FP64,
V128
}
static class OperandTypeExtensions
{
public static bool IsInteger(this OperandType type)
{
return type == OperandType.I32 ||
type == OperandType.I64;
}
public static RegisterType ToRegisterType(this OperandType type)
{
switch (type)
{
case OperandType.FP32: return RegisterType.Vector;
case OperandType.FP64: return RegisterType.Vector;
case OperandType.I32: return RegisterType.Integer;
case OperandType.I64: return RegisterType.Integer;
case OperandType.V128: return RegisterType.Vector;
}
throw new InvalidOperationException($"Invalid operand type \"{type}\".");
}
public static int GetSizeInBytes(this OperandType type)
{
switch (type)
{
case OperandType.FP32: return 4;
case OperandType.FP64: return 8;
case OperandType.I32: return 4;
case OperandType.I64: return 8;
case OperandType.V128: return 16;
}
throw new InvalidOperationException($"Invalid operand type \"{type}\".");
}
}
}

40
ARMeilleure/IntermediateRepresentation/Operation.cs Normal file
View file

@ -0,0 +1,40 @@
namespace ARMeilleure.IntermediateRepresentation
{
class Operation : Node
{
public Instruction Instruction { get; private set; }
public Operation(
Instruction instruction,
Operand destination,
params Operand[] sources) : base(destination, sources.Length)
{
Instruction = instruction;
for (int index = 0; index < sources.Length; index++)
{
SetSource(index, sources[index]);
}
}
public Operation(
Instruction instruction,
Operand[] destinations,
Operand[] sources) : base(destinations, sources.Length)
{
Instruction = instruction;
for (int index = 0; index < sources.Length; index++)
{
SetSource(index, sources[index]);
}
}
public void TurnIntoCopy(Operand source)
{
Instruction = Instruction.Copy;
SetSources(new Operand[] { source });
}
}
}

22
ARMeilleure/IntermediateRepresentation/PhiNode.cs Normal file
View file

@ -0,0 +1,22 @@
namespace ARMeilleure.IntermediateRepresentation
{
class PhiNode : Node
{
private BasicBlock[] _blocks;
public PhiNode(Operand destination, int predecessorsCount) : base(destination, predecessorsCount)
{
_blocks = new BasicBlock[predecessorsCount];
}
public BasicBlock GetBlock(int index)
{
return _blocks[index];
}
public void SetBlock(int index, BasicBlock block)
{
_blocks[index] = block;
}
}
}

43
ARMeilleure/IntermediateRepresentation/Register.cs Normal file
View file

@ -0,0 +1,43 @@
using System;
namespace ARMeilleure.IntermediateRepresentation
{
struct Register : IEquatable<Register>
{
public int Index { get; }
public RegisterType Type { get; }
public Register(int index, RegisterType type)
{
Index = index;
Type = type;
}
public override int GetHashCode()
{
return (ushort)Index | ((int)Type << 16);
}
public static bool operator ==(Register x, Register y)
{
return x.Equals(y);
}
public static bool operator !=(Register x, Register y)
{
return !x.Equals(y);
}
public override bool Equals(object obj)
{
return obj is Register reg && Equals(reg);
}
public bool Equals(Register other)
{
return other.Index == Index &&
other.Type == Type;
}
}
}

9
ARMeilleure/IntermediateRepresentation/RegisterType.cs Normal file
View file

@ -0,0 +1,9 @@
namespace ARMeilleure.IntermediateRepresentation
{
enum RegisterType
{
Integer,
Vector,
Flag
}
}