Mirrors/N64Recomp
1
0
Fork 0
mirror of https://github.com/N64Recomp/N64Recomp.git synced 2025-07-14 21:55:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Modding Support PR 1 (Instruction tables, modding support, mod symbol format, library conversion) (#89)

Browse source 
* Initial implementation of binary operation table

* Initial implementation of unary operation table

* More binary op types, moved binary expression string generation into separate function

* Added and implemented conditional branch instruction table

* Fixed likely swap on bgezal, fixed extra indent branch close and missing
indent on branch statement

* Add operands for other uses of float registers

* Added CHECK_FR generation to binary operation processing, moved float comparison instructions to binary op table

* Finished moving float arithmetic instructions to operation tables

* Added store instruction operation table

* Created Generator interface, separated operation types and tables and C generation code into new files

* Fix mov.d using the wrong input operand

* Move recompiler core logic into a core library and make the existing CLI consume the core library

* Removed unnecessary config input to recompilation functions

* Moved parts of recomp_port.h into new internal headers in src folder

* Changed recomp port naming to N64Recomp

* Remove some unused code and document which Context fields are actually required for recompilation

* Implement mod symbol parsing

* Restructure mod symbols to make replacements global instead of per-section

* Refactor elf parsing into static Context method for reusability

* Move elf parsing into a separate library

* WIP elf to mod tool, currently working without relocations or API exports/imports

* Make mod tool emit relocs and patch binary for non-relocatable symbol references as needed

* Implemented writing import and exports in the mod tool

* Add dependencies to the mod symbol format, finish exporting and importing of mod symbols

* Add first pass offline mod recompiler (generates C from mods that can be compiled and linked into a dynamic library)

* Add strict mode and ability to generate exports for normal recompilation (for patches)

* Move mod context fields into base context, move import symbols into separate vector, misc cleanup

* Some cleanup by making some Context members private

* Add events (from dependencies and exported) and callbacks to the mod symbol format and add support to them in elf parsing

* Add runtime-driven fields to offline mod recompiler, fix event symbol relocs using the wrong section in the mod tool

* Move file header writing outside of function recompilation

* Allow cross-section relocations, encode exact target section in mod relocations, add way to tag reference symbol relocations

* Add local symbol addresses array to offline mod recompiler output and rename original one to reference section addresses

* Add more comments to the offline mod recompiler's output

* Fix handling of section load addresses to match objcopy behavior, added event parsing to dependency tomls, minor cleanup

* Fixed incorrect size used for finding section segments

* Add missing includes for libstdc++

* Rework callbacks and imports to use the section name for identifying the dependency instead of relying on per-dependency tomls
This commit is contained in:
Wiseguy 2024-08-26 23:06:34 -04:00 • committed by GitHub
parent f8d439aeee
commit 5b17bf8bb5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 5121 additions and 2515 deletions
Download patch file Download diff file Expand all files Collapse all files

56
include/generator.h Normal file
View file

@ -0,0 +1,56 @@
#ifndef __GENERATOR_H__
#define __GENERATOR_H__
#include "n64recomp.h"
#include "operations.h"
namespace N64Recomp {
struct InstructionContext {
int rd;
int rs;
int rt;
int sa;
int fd;
int fs;
int ft;
int cop1_cs;
uint16_t imm16;
bool reloc_tag_as_reference;
RelocType reloc_type;
uint32_t reloc_section_index;
uint32_t reloc_target_section_offset;
};
class Generator {
public:
virtual void process_binary_op(std::ostream& output_file, const BinaryOp& op, const InstructionContext& ctx) const = 0;
virtual void process_unary_op(std::ostream& output_file, const UnaryOp& op, const InstructionContext& ctx) const = 0;
virtual void process_store_op(std::ostream& output_file, const StoreOp& op, const InstructionContext& ctx) const = 0;
virtual void emit_branch_condition(std::ostream& output_file, const ConditionalBranchOp& op, const InstructionContext& ctx) const = 0;
virtual void emit_branch_close(std::ostream& output_file) const = 0;
virtual void emit_check_fr(std::ostream& output_file, int fpr) const = 0;
virtual void emit_check_nan(std::ostream& output_file, int fpr, bool is_double) const = 0;
};
class CGenerator final : Generator {
public:
CGenerator() = default;
void process_binary_op(std::ostream& output_file, const BinaryOp& op, const InstructionContext& ctx) const final;
void process_unary_op(std::ostream& output_file, const UnaryOp& op, const InstructionContext& ctx) const final;
void process_store_op(std::ostream& output_file, const StoreOp& op, const InstructionContext& ctx) const final;
void emit_branch_condition(std::ostream& output_file, const ConditionalBranchOp& op, const InstructionContext& ctx) const final;
void emit_branch_close(std::ostream& output_file) const final;
void emit_check_fr(std::ostream& output_file, int fpr) const final;
void emit_check_nan(std::ostream& output_file, int fpr, bool is_double) const final;
private:
void get_operand_string(Operand operand, UnaryOpType operation, const InstructionContext& context, std::string& operand_string) const;
void get_binary_expr_string(BinaryOpType type, const BinaryOperands& operands, const InstructionContext& ctx, const std::string& output, std::string& expr_string) const;
void get_notation(BinaryOpType op_type, std::string& func_string, std::string& infix_string) const;
};
}
#endif

556
include/n64recomp.h Normal file
View file

@ -0,0 +1,556 @@
#ifndef __RECOMP_PORT__
#define __RECOMP_PORT__
#include <span>
#include <string_view>
#include <cstdint>
#include <utility>
#include <vector>
#include <unordered_map>
#include <unordered_set>
#include <filesystem>
#ifdef _MSC_VER
inline uint32_t byteswap(uint32_t val) {
return _byteswap_ulong(val);
}
#else
constexpr uint32_t byteswap(uint32_t val) {
return __builtin_bswap32(val);
}
#endif
namespace N64Recomp {
struct Function {
uint32_t vram;
uint32_t rom;
std::vector<uint32_t> words;
std::string name;
uint16_t section_index;
bool ignored;
bool reimplemented;
bool stubbed;
std::unordered_map<int32_t, std::string> function_hooks;
Function(uint32_t vram, uint32_t rom, std::vector<uint32_t> words, std::string name, uint16_t section_index, bool ignored = false, bool reimplemented = false, bool stubbed = false)
: vram(vram), rom(rom), words(std::move(words)), name(std::move(name)), section_index(section_index), ignored(ignored), reimplemented(reimplemented), stubbed(stubbed) {}
Function() = default;
};
enum class RelocType : uint8_t {
R_MIPS_NONE = 0,
R_MIPS_16,
R_MIPS_32,
R_MIPS_REL32,
R_MIPS_26,
R_MIPS_HI16,
R_MIPS_LO16,
R_MIPS_GPREL16,
};
struct Reloc {
uint32_t address;
uint32_t target_section_offset;
uint32_t symbol_index; // Only used for reference symbols and special section symbols
uint16_t target_section;
RelocType type;
bool reference_symbol;
};
// Special section indices.
constexpr uint16_t SectionAbsolute = (uint16_t)-2;
constexpr uint16_t SectionImport = (uint16_t)-3; // Imported symbols for mods
constexpr uint16_t SectionEvent = (uint16_t)-4;
// Special section names.
constexpr std::string_view PatchSectionName = ".recomp_patch";
constexpr std::string_view ForcedPatchSectionName = ".recomp_force_patch";
constexpr std::string_view ExportSectionName = ".recomp_export";
constexpr std::string_view EventSectionName = ".recomp_event";
constexpr std::string_view ImportSectionPrefix = ".recomp_import.";
constexpr std::string_view CallbackSectionPrefix = ".recomp_callback.";
// Special mod names.
constexpr std::string_view ModSelf = ".";
constexpr std::string_view ModBaseRecomp = "*";
struct Section {
uint32_t rom_addr = 0;
uint32_t ram_addr = 0;
uint32_t size = 0;
uint32_t bss_size = 0; // not populated when using a symbol toml
std::vector<uint32_t> function_addrs; // only used by the CLI (to find the size of static functions)
std::vector<Reloc> relocs;
std::string name;
uint16_t bss_section_index = (uint16_t)-1;
bool executable = false;
bool relocatable = false; // TODO is this needed? relocs being non-empty should be an equivalent check.
bool has_mips32_relocs = false;
};
struct ReferenceSection {
uint32_t rom_addr;
uint32_t ram_addr;
uint32_t size;
bool relocatable;
};
struct ReferenceSymbol {
std::string name;
uint16_t section_index;
uint32_t section_offset;
bool is_function;
};
struct ElfParsingConfig {
std::string bss_section_suffix;
// Functions with manual size overrides
std::unordered_map<std::string, size_t> manually_sized_funcs;
// The section names that were specified as relocatable
std::unordered_set<std::string> relocatable_sections;
bool has_entrypoint;
int32_t entrypoint_address;
bool use_absolute_symbols;
bool unpaired_lo16_warnings;
bool all_sections_relocatable;
};
struct DataSymbol {
uint32_t vram;
std::string name;
DataSymbol(uint32_t vram, std::string&& name) : vram(vram), name(std::move(name)) {}
};
using DataSymbolMap = std::unordered_map<uint16_t, std::vector<DataSymbol>>;
extern const std::unordered_set<std::string> reimplemented_funcs;
extern const std::unordered_set<std::string> ignored_funcs;
extern const std::unordered_set<std::string> renamed_funcs;
struct Dependency {
uint8_t major_version;
uint8_t minor_version;
uint8_t patch_version;
std::string mod_id;
};
struct ImportSymbol {
ReferenceSymbol base;
size_t dependency_index;
};
struct DependencyEvent {
size_t dependency_index;
std::string event_name;
};
struct EventSymbol {
ReferenceSymbol base;
};
struct Callback {
size_t function_index;
size_t dependency_event_index;
};
struct SymbolReference {
// Reference symbol section index, or one of the special section indices such as SectionImport.
uint16_t section_index;
size_t symbol_index;
};
enum class ReplacementFlags : uint32_t {
Force = 1 << 0,
};
inline ReplacementFlags operator&(ReplacementFlags lhs, ReplacementFlags rhs) { return ReplacementFlags(uint32_t(lhs) & uint32_t(rhs)); }
inline ReplacementFlags operator|(ReplacementFlags lhs, ReplacementFlags rhs) { return ReplacementFlags(uint32_t(lhs) | uint32_t(rhs)); }
struct FunctionReplacement {
uint32_t func_index;
uint32_t original_section_vrom;
uint32_t original_vram;
ReplacementFlags flags;
};
class Context {
private:
//// Reference symbols (used for populating relocations for patches)
// A list of the sections that contain the reference symbols.
std::vector<ReferenceSection> reference_sections;
// A list of the reference symbols.
std::vector<ReferenceSymbol> reference_symbols;
// Mapping of symbol name to reference symbol index.
std::unordered_map<std::string, SymbolReference> reference_symbols_by_name;
public:
std::vector<Section> sections;
std::vector<Function> functions;
// A list of the list of each function (by index in `functions`) in a given section
std::vector<std::vector<size_t>> section_functions;
// A mapping of vram address to every function with that address.
std::unordered_map<uint32_t, std::vector<size_t>> functions_by_vram;
// A mapping of bss section index to the corresponding non-bss section index.
std::unordered_map<uint16_t, uint16_t> bss_section_to_section;
// The target ROM being recompiled, TODO move this outside of the context to avoid making a copy for mod contexts.
// Used for reading relocations and for the output binary feature.
std::vector<uint8_t> rom;
//// Only used by the CLI, TODO move this to a struct in the internal headers.
// A mapping of function name to index in the functions vector
std::unordered_map<std::string, size_t> functions_by_name;
//// Mod dependencies and their symbols
//// Imported values
// List of dependencies.
std::vector<Dependency> dependencies;
// Mapping of dependency name to dependency index.
std::unordered_map<std::string, size_t> dependencies_by_name;
// List of symbols imported from dependencies.
std::vector<ImportSymbol> import_symbols;
// List of events imported from dependencies.
std::vector<DependencyEvent> dependency_events;
// Mappings of dependency event name to the index in dependency_events, all indexed by dependency.
std::vector<std::unordered_map<std::string, size_t>> dependency_events_by_name;
// Mappings of dependency import name to index in import_symbols, all indexed by dependency.
std::vector<std::unordered_map<std::string, size_t>> dependency_imports_by_name;
//// Exported values
// List of function replacements, which contains the original function to replace and the function index to replace it with.
std::vector<FunctionReplacement> replacements;
// Indices of every exported function.
std::vector<size_t> exported_funcs;
// List of callbacks, which contains the function for the callback and the dependency event it attaches to.
std::vector<Callback> callbacks;
// List of symbols from events, which contains the names of events that this context provides.
std::vector<EventSymbol> event_symbols;
// Imports sections and function symbols from a provided context into this context's reference sections and reference functions.
bool import_reference_context(const Context& reference_context);
// Reads a data symbol file and adds its contents into this context's reference data symbols.
bool read_data_reference_syms(const std::filesystem::path& data_syms_file_path);
static bool from_symbol_file(const std::filesystem::path& symbol_file_path, std::vector<uint8_t>&& rom, Context& out, bool with_relocs);
static bool from_elf_file(const std::filesystem::path& elf_file_path, Context& out, const ElfParsingConfig& flags, bool for_dumping_context, DataSymbolMap& data_syms_out, bool& found_entrypoint_out);
Context() = default;
bool add_dependency(const std::string& id, uint8_t major_version, uint8_t minor_version, uint8_t patch_version) {
if (dependencies_by_name.contains(id)) {
return false;
}
size_t dependency_index = dependencies.size();
dependencies.emplace_back(N64Recomp::Dependency {
.major_version = major_version,
.minor_version = minor_version,
.patch_version = patch_version,
.mod_id = id
});
dependencies_by_name.emplace(id, dependency_index);
dependency_events_by_name.resize(dependencies.size());
dependency_imports_by_name.resize(dependencies.size());
return true;
}
bool add_dependencies(const std::vector<Dependency>& new_dependencies) {
dependencies.reserve(dependencies.size() + new_dependencies.size());
dependencies_by_name.reserve(dependencies_by_name.size() + new_dependencies.size());
// Check if any of the dependencies already exist and fail if so.
for (const Dependency& dep : new_dependencies) {
if (dependencies_by_name.contains(dep.mod_id)) {
return false;
}
}
for (const Dependency& dep : new_dependencies) {
size_t dependency_index = dependencies.size();
dependencies.emplace_back(dep);
dependencies_by_name.emplace(dep.mod_id, dependency_index);
}
dependency_events_by_name.resize(dependencies.size());
dependency_imports_by_name.resize(dependencies.size());
return true;
}
bool find_dependency(const std::string& mod_id, size_t& dependency_index) {
auto find_it = dependencies_by_name.find(mod_id);
if (find_it == dependencies_by_name.end()) {
return false;
}
dependency_index = find_it->second;
return true;
}
size_t find_function_by_vram_section(uint32_t vram, size_t section_index) const {
auto find_it = functions_by_vram.find(vram);
if (find_it == functions_by_vram.end()) {
return (size_t)-1;
}
for (size_t function_index : find_it->second) {
if (functions[function_index].section_index == section_index) {
return function_index;
}
}
return (size_t)-1;
}
bool has_reference_symbols() const {
return !reference_symbols.empty() || !import_symbols.empty() || !event_symbols.empty();
}
bool is_regular_reference_section(uint16_t section_index) const {
return section_index != SectionImport && section_index != SectionEvent;
}
bool find_reference_symbol(const std::string& symbol_name, SymbolReference& ref_out) const {
auto find_sym_it = reference_symbols_by_name.find(symbol_name);
// Check if the symbol was found.
if (find_sym_it == reference_symbols_by_name.end()) {
return false;
}
ref_out = find_sym_it->second;
return true;
}
bool reference_symbol_exists(const std::string& symbol_name) const {
SymbolReference dummy_ref;
return find_reference_symbol(symbol_name, dummy_ref);
}
bool find_regular_reference_symbol(const std::string& symbol_name, SymbolReference& ref_out) const {
SymbolReference ref_found;
if (!find_reference_symbol(symbol_name, ref_found)) {
return false;
}
// Ignore reference symbols in special sections.
if (!is_regular_reference_section(ref_found.section_index)) {
return false;
}
ref_out = ref_found;
return true;
}
const ReferenceSymbol& get_reference_symbol(uint16_t section_index, size_t symbol_index) const {
if (section_index == SectionImport) {
return import_symbols[symbol_index].base;
}
else if (section_index == SectionEvent) {
return event_symbols[symbol_index].base;
}
return reference_symbols[symbol_index];
}
size_t num_regular_reference_symbols() {
return reference_symbols.size();
}
const ReferenceSymbol& get_regular_reference_symbol(size_t index) const {
return reference_symbols[index];
}
const ReferenceSymbol& get_reference_symbol(const SymbolReference& ref) const {
return get_reference_symbol(ref.section_index, ref.symbol_index);
}
bool is_reference_section_relocatable(uint16_t section_index) const {
if (section_index == SectionAbsolute) {
return false;
}
else if (section_index == SectionImport || section_index == SectionEvent) {
return true;
}
return reference_sections[section_index].relocatable;
}
bool add_reference_symbol(const std::string& symbol_name, uint16_t section_index, uint32_t vram, bool is_function) {
uint32_t section_vram;
if (section_index == SectionAbsolute) {
section_vram = 0;
}
else if (section_index < reference_sections.size()) {
section_vram = reference_sections[section_index].ram_addr;
}
// Invalid section index.
else {
return false;
}
// TODO Check if reference_symbols_by_name already contains the name and show a conflict error if so.
reference_symbols_by_name.emplace(symbol_name, N64Recomp::SymbolReference{
.section_index = section_index,
.symbol_index = reference_symbols.size()
});
reference_symbols.emplace_back(N64Recomp::ReferenceSymbol{
.name = symbol_name,
.section_index = section_index,
.section_offset = vram - section_vram,
.is_function = is_function
});
return true;
}
void add_import_symbol(const std::string& symbol_name, size_t dependency_index) {
// TODO Check if dependency_imports_by_name[dependency_index] already contains the name and show a conflict error if so.
dependency_imports_by_name[dependency_index][symbol_name] = import_symbols.size();
import_symbols.emplace_back(
N64Recomp::ImportSymbol {
.base = N64Recomp::ReferenceSymbol {
.name = symbol_name,
.section_index = N64Recomp::SectionImport,
.section_offset = 0,
.is_function = true
},
.dependency_index = dependency_index,
}
);
}
bool find_import_symbol(const std::string& symbol_name, size_t dependency_index, SymbolReference& ref_out) const {
if (dependency_index >= dependencies.size()) {
return false;
}
auto find_it = dependency_imports_by_name[dependency_index].find(symbol_name);
if (find_it == dependency_imports_by_name[dependency_index].end()) {
return false;
}
ref_out.section_index = SectionImport;
ref_out.symbol_index = find_it->second;
return true;
}
void add_event_symbol(const std::string& symbol_name) {
// TODO Check if reference_symbols_by_name already contains the name and show a conflict error if so.
reference_symbols_by_name[symbol_name] = N64Recomp::SymbolReference {
.section_index = N64Recomp::SectionEvent,
.symbol_index = event_symbols.size()
};
event_symbols.emplace_back(
N64Recomp::EventSymbol {
.base = N64Recomp::ReferenceSymbol {
.name = symbol_name,
.section_index = N64Recomp::SectionEvent,
.section_offset = 0,
.is_function = true
}
}
);
}
bool find_event_symbol(const std::string& symbol_name, SymbolReference& ref_out) const {
SymbolReference ref_found;
if (!find_reference_symbol(symbol_name, ref_found)) {
return false;
}
// Ignore reference symbols that aren't in the event section.
if (ref_found.section_index != SectionEvent) {
return false;
}
ref_out = ref_found;
return true;
}
bool add_dependency_event(const std::string& event_name, size_t dependency_index, size_t& dependency_event_index) {
if (dependency_index >= dependencies.size()) {
return false;
}
// Prevent adding the same event to a dependency twice. This isn't an error, since a mod could register
// multiple callbacks to the same event.
auto find_it = dependency_events_by_name[dependency_index].find(event_name);
if (find_it != dependency_events_by_name[dependency_index].end()) {
dependency_event_index = find_it->second;
return true;
}
dependency_event_index = dependency_events.size();
dependency_events.emplace_back(DependencyEvent{
.dependency_index = dependency_index,
.event_name = event_name
});
dependency_events_by_name[dependency_index][event_name] = dependency_event_index;
return true;
}
bool add_callback(size_t dependency_event_index, size_t function_index) {
callbacks.emplace_back(Callback{
.function_index = function_index,
.dependency_event_index = dependency_event_index
});
return true;
}
uint32_t get_reference_section_vram(uint16_t section_index) const {
if (section_index == N64Recomp::SectionAbsolute) {
return 0;
}
else if (!is_regular_reference_section(section_index)) {
return 0;
}
else {
return reference_sections[section_index].ram_addr;
}
}
uint32_t get_reference_section_rom(uint16_t section_index) const {
if (section_index == N64Recomp::SectionAbsolute) {
return (uint32_t)-1;
}
else if (!is_regular_reference_section(section_index)) {
return (uint32_t)-1;
}
else {
return reference_sections[section_index].rom_addr;
}
}
void copy_reference_sections_from(const Context& rhs) {
reference_sections = rhs.reference_sections;
}
};
bool recompile_function(const Context& context, const Function& func, std::ofstream& output_file, std::span<std::vector<uint32_t>> static_funcs, bool tag_reference_relocs);
enum class ModSymbolsError {
Good,
NotASymbolFile,
UnknownSymbolFileVersion,
CorruptSymbolFile,
FunctionOutOfBounds,
};
ModSymbolsError parse_mod_symbols(std::span<const char> data, std::span<const uint8_t> binary, const std::unordered_map<uint32_t, uint16_t>& sections_by_vrom, const Context& reference_context, Context& context_out);
std::vector<uint8_t> symbols_to_bin_v1(const Context& mod_context);
inline bool validate_mod_name(std::string_view str) {
// Disallow mod names with a colon in them, since you can't specify that in a dependency string orin callbacks.
for (char c : str) {
if (c == ':') {
return false;
}
}
return true;
}
inline bool validate_mod_name(const std::string& str) {
return validate_mod_name(std::string_view{str});
}
}
#endif

200
include/operations.h Normal file
View file

@ -0,0 +1,200 @@
#ifndef __OPERATIONS_H__
#define __OPERATIONS_H__
#include <unordered_map>
#include "rabbitizer.hpp"
namespace N64Recomp {
using InstrId = rabbitizer::InstrId::UniqueId;
using Cop0Reg = rabbitizer::Registers::Cpu::Cop0;
enum class StoreOpType {
SD,
SDL,
SDR,
SW,
SWL,
SWR,
SH,
SB,
SDC1,
SWC1
};
enum class UnaryOpType {
None,
ToS32,
ToU32,
ToS64,
ToU64,
NegateS32,
NegateS64,
Lui,
Mask5, // Mask to 5 bits
Mask6, // Mask to 5 bits
ToInt32, // Functionally equivalent to ToS32, only exists for parity with old codegen
Negate,
AbsFloat,
AbsDouble,
SqrtFloat,
SqrtDouble,
ConvertSFromW,
ConvertWFromS,
ConvertDFromW,
ConvertWFromD,
ConvertDFromS,
ConvertSFromD,
ConvertDFromL,
ConvertLFromD,
ConvertSFromL,
ConvertLFromS,
TruncateWFromS,
TruncateWFromD,
RoundWFromS,
RoundWFromD,
CeilWFromS,
CeilWFromD,
FloorWFromS,
FloorWFromD
};
enum class BinaryOpType {
// Addition/subtraction
Add32,
Sub32,
Add64,
Sub64,
// Float arithmetic
AddFloat,
AddDouble,
SubFloat,
SubDouble,
MulFloat,
MulDouble,
DivFloat,
DivDouble,
// Bitwise
And64,
Or64,
Nor64,
Xor64,
Sll32,
Sll64,
Srl32,
Srl64,
Sra32,
Sra64,
// Comparisons
Equal,
NotEqual,
Less,
LessEq,
Greater,
GreaterEq,
// Loads
LD,
LW,
LWU,
LH,
LHU,
LB,
LBU,
LDL,
LDR,
LWL,
LWR,
// Fixed result
True,
False,
COUNT,
};
enum class Operand {
Rd, // GPR
Rs, // GPR
Rt, // GPR
Fd, // FPR
Fs, // FPR
Ft, // FPR
FdDouble, // Double float in fd FPR
FsDouble, // Double float in fs FPR
FtDouble, // Double float in ft FPR
// Raw low 32-bit values of FPRs with handling for mips3 float mode behavior
FdU32L,
FsU32L,
FtU32L,
// Raw high 32-bit values of FPRs with handling for mips3 float mode behavior
FdU32H,
FsU32H,
FtU32H,
// Raw 64-bit values of FPRs
FdU64,
FsU64,
FtU64,
ImmU16, // 16-bit immediate, unsigned
ImmS16, // 16-bit immediate, signed
Sa, // Shift amount
Sa32, // Shift amount plus 32
Cop1cs, // Coprocessor 1 Condition Signal
Hi,
Lo,
Zero,
Base = Rs, // Alias for Rs for loads
};
struct StoreOp {
StoreOpType type;
Operand value_input;
};
struct UnaryOp {
UnaryOpType operation;
Operand output;
Operand input;
// Whether the FR bit needs to be checked for odd float registers for this instruction.
bool check_fr = false;
// Whether the input need to be checked for being NaN.
bool check_nan = false;
};
struct BinaryOperands {
// Operation to apply to each operand before applying the binary operation to them.
UnaryOpType operand_operations[2];
// The source of the input operands.
Operand operands[2];
};
struct BinaryOp {
// The type of binary operation this represents.
BinaryOpType type;
// The output operand.
Operand output;
// The input operands.
BinaryOperands operands;
// Whether the FR bit needs to be checked for odd float registers for this instruction.
bool check_fr = false;
// Whether the inputs need to be checked for being NaN.
bool check_nan = false;
};
struct ConditionalBranchOp {
// The type of binary operation to use for this compare
BinaryOpType comparison;
// The input operands.
BinaryOperands operands;
// Whether this jump should link for returns.
bool link;
// Whether this jump has "likely" behavior (doesn't execute the delay slot if skipped).
bool likely;
};
extern const std::unordered_map<InstrId, UnaryOp> unary_ops;
extern const std::unordered_map<InstrId, BinaryOp> binary_ops;
extern const std::unordered_map<InstrId, ConditionalBranchOp> conditional_branch_ops;
extern const std::unordered_map<InstrId, StoreOp> store_ops;
}
#endif

236
include/recomp_port.h
View file

@ -1,236 +0,0 @@
#ifndef __RECOMP_PORT__
#define __RECOMP_PORT__
#include <span>
#include <string_view>
#include <cstdint>
#include <utility>
#include <vector>
#include <unordered_map>
#include <span>
#include <unordered_set>
#include <filesystem>
#include "rabbitizer.hpp"
#include "elfio/elfio.hpp"
#ifdef _MSC_VER
inline uint32_t byteswap(uint32_t val) {
return _byteswap_ulong(val);
}
#else
constexpr uint32_t byteswap(uint32_t val) {
return __builtin_bswap32(val);
}
#endif
namespace RecompPort {
// Potential argument types for function declarations
enum class FunctionArgType {
u32,
s32,
};
// Mapping of function name to argument types
using DeclaredFunctionMap = std::unordered_map<std::string, std::vector<FunctionArgType>>;
struct InstructionPatch {
std::string func_name;
int32_t vram;
uint32_t value;
};
struct FunctionHook {
std::string func_name;
int32_t before_vram;
std::string text;
};
struct FunctionSize {
std::string func_name;
uint32_t size_bytes;
FunctionSize(const std::string& func_name, uint32_t size_bytes) : func_name(std::move(func_name)), size_bytes(size_bytes) {}
};
struct ManualFunction {
std::string func_name;
std::string section_name;
uint32_t vram;
uint32_t size;
ManualFunction(const std::string& func_name, std::string section_name, uint32_t vram, uint32_t size) : func_name(std::move(func_name)), section_name(std::move(section_name)), vram(vram), size(size) {}
};
struct Config {
int32_t entrypoint;
int32_t functions_per_output_file;
bool has_entrypoint;
bool uses_mips3_float_mode;
bool single_file_output;
bool use_absolute_symbols;
bool unpaired_lo16_warnings;
std::filesystem::path elf_path;
std::filesystem::path symbols_file_path;
std::filesystem::path func_reference_syms_file_path;
std::vector<std::filesystem::path> data_reference_syms_file_paths;
std::filesystem::path rom_file_path;
std::filesystem::path output_func_path;
std::filesystem::path relocatable_sections_path;
std::filesystem::path output_binary_path;
std::vector<std::string> stubbed_funcs;
std::vector<std::string> ignored_funcs;
DeclaredFunctionMap declared_funcs;
std::vector<InstructionPatch> instruction_patches;
std::vector<FunctionHook> function_hooks;
std::vector<FunctionSize> manual_func_sizes;
std::vector<ManualFunction> manual_functions;
std::string bss_section_suffix;
std::string recomp_include;
Config(const char* path);
bool good() { return !bad; }
private:
bool bad;
};
struct JumpTable {
uint32_t vram;
uint32_t addend_reg;
uint32_t rom;
uint32_t lw_vram;
uint32_t addu_vram;
uint32_t jr_vram;
std::vector<uint32_t> entries;
JumpTable(uint32_t vram, uint32_t addend_reg, uint32_t rom, uint32_t lw_vram, uint32_t addu_vram, uint32_t jr_vram, std::vector<uint32_t>&& entries)
: vram(vram), addend_reg(addend_reg), rom(rom), lw_vram(lw_vram), addu_vram(addu_vram), jr_vram(jr_vram), entries(std::move(entries)) {}
};
struct AbsoluteJump {
uint32_t jump_target;
uint32_t instruction_vram;
AbsoluteJump(uint32_t jump_target, uint32_t instruction_vram) : jump_target(jump_target), instruction_vram(instruction_vram) {}
};
struct Function {
uint32_t vram;
uint32_t rom;
std::vector<uint32_t> words;
std::string name;
ELFIO::Elf_Half section_index;
bool ignored;
bool reimplemented;
bool stubbed;
std::unordered_map<int32_t, std::string> function_hooks;
Function(uint32_t vram, uint32_t rom, std::vector<uint32_t> words, std::string name, ELFIO::Elf_Half section_index, bool ignored = false, bool reimplemented = false, bool stubbed = false)
: vram(vram), rom(rom), words(std::move(words)), name(std::move(name)), section_index(section_index), ignored(ignored), reimplemented(reimplemented), stubbed(stubbed) {}
Function() = default;
};
enum class RelocType : uint8_t {
R_MIPS_NONE = 0,
R_MIPS_16,
R_MIPS_32,
R_MIPS_REL32,
R_MIPS_26,
R_MIPS_HI16,
R_MIPS_LO16,
R_MIPS_GPREL16,
};
struct Reloc {
uint32_t address;
uint32_t section_offset;
uint32_t symbol_index;
uint32_t target_section;
RelocType type;
bool reference_symbol;
};
constexpr uint16_t SectionSelf = (uint16_t)-1;
constexpr uint16_t SectionAbsolute = (uint16_t)-2;
struct Section {
ELFIO::Elf_Xword rom_addr = 0;
ELFIO::Elf64_Addr ram_addr = 0;
ELFIO::Elf_Xword size = 0;
std::vector<uint32_t> function_addrs;
std::vector<Reloc> relocs;
std::string name;
ELFIO::Elf_Half bss_section_index = (ELFIO::Elf_Half)-1;
bool executable = false;
bool relocatable = false;
bool has_mips32_relocs = false;
};
struct FunctionStats {
std::vector<JumpTable> jump_tables;
std::vector<AbsoluteJump> absolute_jumps;
};
struct ReferenceSection {
uint32_t rom_addr;
uint32_t ram_addr;
uint32_t size;
bool relocatable;
};
struct ReferenceSymbol {
uint16_t section_index;
uint32_t section_offset;
bool is_function;
};
struct Context {
// ROM address of each section
std::vector<Section> sections;
std::vector<Function> functions;
std::unordered_map<uint32_t, std::vector<size_t>> functions_by_vram;
// A mapping of function name to index in the functions vector
std::unordered_map<std::string, size_t> functions_by_name;
std::vector<uint8_t> rom;
// A list of the list of each function (by index in `functions`) in a given section
std::vector<std::vector<size_t>> section_functions;
// The section names that were specified as relocatable
std::unordered_set<std::string> relocatable_sections;
// Functions with manual size overrides
std::unordered_map<std::string, size_t> manually_sized_funcs;
//// Reference symbols (used for populating relocations for patches)
// A list of the sections that contain the reference symbols.
std::vector<ReferenceSection> reference_sections;
// A list of the reference symbols.
std::vector<ReferenceSymbol> reference_symbols;
// Name of every reference symbol in the same order as `reference_symbols`.
std::vector<std::string> reference_symbol_names;
// Mapping of symbol name to reference symbol index.
std::unordered_map<std::string, size_t> reference_symbols_by_name;
int executable_section_count;
Context(const ELFIO::elfio& elf_file) {
sections.resize(elf_file.sections.size());
section_functions.resize(elf_file.sections.size());
functions.reserve(1024);
functions_by_vram.reserve(functions.capacity());
functions_by_name.reserve(functions.capacity());
rom.reserve(8 * 1024 * 1024);
executable_section_count = 0;
}
// Imports sections and function symbols from a provided context into this context's reference sections and reference functions.
void import_reference_context(const Context& reference_context);
// Reads a data symbol file and adds its contents into this context's reference data symbols.
bool read_data_reference_syms(const std::filesystem::path& data_syms_file_path);
static bool from_symbol_file(const std::filesystem::path& symbol_file_path, std::vector<uint8_t>&& rom, Context& out, bool with_relocs);
Context() = default;
};
bool analyze_function(const Context& context, const Function& function, const std::vector<rabbitizer::InstructionCpu>& instructions, FunctionStats& stats);
bool recompile_function(const Context& context, const Config& config, const Function& func, std::ofstream& output_file, std::span<std::vector<uint32_t>> static_funcs, bool write_header);
}
#endif