* Implement sceKernelMemoryPoolBatch
I've tested Commit and Decommit on real hardware, haven't tested Protect or TypeProtect yet.
Implementation is primarily based on our sceKernelBatchMap implementation.
* Clang
* Update sceKernelMemoryPoolExpand
Hardware tests show that this function is basically the same as sceKernelAllocateDirectMemory, with some minor differences.
Update the memory searching code to match my updated AllocateDirectMemory code, with appropriate error conditions.
* Update MemoryPoolReserve
Only difference between real hw and our code is behavior with addr = 0.
* Don't coalesce PoolReserved areas.
Real hardware doesn't coalesce them.
* Update PoolCommit
Plenty of edge case behaviors to handle here.
Addresses are treated as fixed, EINVAL is returned for bad mappings, name should be preserved from PoolReserving, committed areas should coalesce, reserved areas get their phys_base updated
* Formatting
* Adjust fixed PoolReserve path
Hardware tests suggest this will overwrite all VMAs in the range. Run UnmapMemoryImpl on the full area, then reserve. Same logic applies to normal reservations too.
Also adjusts logic of the non-fixed path to more closely align with hardware observations.
* Remove phys_base modifications
This can be handled later. Doing the logic properly would likely take work in MergeAdjacent, and would probably need to be applied to normal dmem mappings too.
* Use VMAHandle.Contains()
Why do extra math when we have a function specifically for this?
* Update memory.cpp
* Remove unnecessary code
Since I've removed those two asserts, these two lines of code effectively do nothing.
* Clang
* Fix names
* Fix PoolDecommit
Should fix the address space regressions in UE titles on Windows.
* Fix error log
Should make the cause of this clearer?
* Clang
* Oops
* Remove coalesce on PoolCommit
Windows makes this more difficult.
* Track pool budgets
If you try to commit more pooled memory than is allocated, PoolCommit returns ENOMEM.
Also fixes error conditions for PoolDecommit, that should return EINVAL if given an address that isn't part of the pool.
Note: Seems like the pool budget can't hit zero? I used a <= comparison based on hardware tests, otherwise we're able to make more mappings than real hardware can.
* Reduce bitfield size
Linux compilers automatically convert this, Windows not so much.
* Static assert for VirtualQueryInfo struct size
Since compilers can be weird, having a static assert for this will be helpful.
Granted, this probably wont need changing after this PR.
* Fix VirtualQuery behavior on low addresses.
* Fix VirtualQuery struct
Somewhere in our BitField and array use, the size of our VirtualQuery struct became larger than the struct used on real hardware.
Fixing this fixes some data corruption visible in the name parameter during my tests.
* Default name to anon
On real hardware, nameless mappings are given the name "anon:address" where address appears to be the address that made the memory call.
For simplicity sake, I'll stick to the name "anon" for now.
* Place an upper bound on returns from SearchFree
Right now, this upper bound is set based on the limitations of our GPU buffer cache and page table.
Someone with more experience in that area of code should probably fix that at some point.
* More anons
* Clang
* Fix name in sceKernelMapNamedDirectMemory
* strncpy instead of strcpy
Hardcoded the constant size for now, I need to review how real hardware behaves here to determine if anything else is necessary for this to be accurate.
* Fix name behavior
All memory naming functions restrict the name size to a 31 character limit, and return `ORBIS_KERNEL_ERROR_ENAMETOOLONG` if that limit is exceeded.
Since this value is constant for all functions involving names, I've defined it as a constant in kernel's memory.h, and used that in place of any hardcoded 32 character limits.
* Error logging
Hopefully this helps in catching the UFC regression?
* Increase address space upper bound
Probably needs heavy testing, especially on Mac/Windows.
This increases the address space, as needed to accommodate strange memory behaviors seen in UFC.
* VirtualQuery fix
Due to limitations of certain platforms, we initialize our vma_map with 3 separate free mappings.
As such, we need to use a while loop here to accurately query mappings with high addresses
* Fix mappings to high addresses
The PS4's GPU can only handle 40bit addresses. Our texture cache and buffer cache were designed around these limits, and mapping to higher addresses would cause segmentation faults and access violations.
To fix these crashes, only map to the GPU if the mapping is fully contained within the address space the GPU should access.
I'm open to suggestions on how to make this cleaner
* Revert "Increase address space upper bound"
This reverts commit 3d50eeeebb.
* Revert VirtualQuery while loop
Windows wasn't happy with this, again.
Will try to debug and properly fix this when I have a good chance.
* Fix asserts
FindVMA, due to the way it's programmed, never actually returns vma_map.end(), the furthest it ever returns is the last valid memory area. All those asserts we involving vma_map.end() never actually trigger due to this.
This commit removes redundant asserts, adds messages to asserts that were lacking them, and fixes all asserts designed to detect out of bounds memory accesses so they actually trigger.
I've also fixed some potential memory safety issues.
* Proper error behavior in QueryProtection
Might as well handle this properly while I'm here.
* Clang
* More information about ReserveVirtualRange results
Should help debug issues like the one in The Order: 1886 (CUSA00076)
* Fix assert message
* Update assert message
Extra space
* Fix my bug
Oh hey, finally something that's my fault.
* Fix rasterizer unmaps
Should use adjusted_size here, otherwise we could unmap too much.
Thanks to diegolix29 for spotting this.
* Fix edge case in MapMemory
Code comments explain everything.
This should fix some memory asserts.
* Fix fix
Avoid running the code path if it's unnecessary, since there are many additional edge cases to handle when the VMA map is small.
* Fix fix fix
Should prevent infinite loops, haven't tested properly yet though.
* Split logging for inputs and out_addr in ReserveVirtualRange
Addresses review comments.
* Proper handling of whence 3 & 4
* Accurate directory handling in open
Directories can be opened, and can be created in open, these changes should handle that more accurately.
* Mount /app0 as read only
On real hardware, it's read only.
* Proper directory flag handling.
Even when directory is specified, it will still succeed to open non-directories.
* Check for read only directories
* Earlier ro check in posix_rmdir
Hardware tests suggest these checks are in a different order
* Clear temp folder on boot
My tests rely on this, and some games do too.
Two birds with one stone
* Clang
* Add missing DeleteHandle calls
Whoops
* Final flags adjustment in sceKernelOpen
All my current tests are now hardware accurate.
* Fix truncates
Host ftruncate consistently fails on EINVAL, I'll need to test if this issue affected Windows too.
* Windows hacks
Windows is more limiting about how folders are opened and things like that. For now, pretend these calls didn't error.
Also fixes compilation for Windows
* Final touch-ups
After expanding my test suite further, I found a couple more edge cases that needed addressing.
Bloodborne audio is still broken, I'll look into that soon.
* Remove hacky read-only behavior in posix_stat
Bloodborne apparently uses the mode parameter here when querying it's audio files, and the mode we returned led to it disabling audio entirely.
* Clang
* Cleaner code
* Combine fsync and sync flags
According to FreeBSD docs, the "sync" flag is synonymous with the fsync flag, and is only included to meet the POSIX spec.
* Log if any currently unhandled flags are encountered.
These are rare and probably not too important, but log a warning when they're seen.
* Update file_system.cpp
* Update file_system.cpp
* Clang
* Revert truncate fix
Using ftruncate works fine after moving the call to before the proper file opening code.
* Truncate before open
Open the file as read-write, then try truncating.
This fixes read | truncate flag behavior on Windows.
* Slightly adjust check for invalid flags
Any open call with invalid flags should return EINVAL, regardless of other errors parameters might cause.
* Fix GetModule exception
Simple mistake
* Prevent OOB writes in add_segment
Due to mistakes in our linker logic, OpenOrbis' libSceFios2 causes OOB writes here.
While the ideal solution would be to fix the erroneous behavior, the best I'm capable of right now is just preventing the OOB writes.
* Implement sceKernelGetModuleInfo, sceKernelGetModuleInfoInternal, sceKernelGetModuleList
These are implemented based on hardware observations and a homebrew sample made by red_prig. I've yet to test what error cases can show up.
* Clang
* Accurate error returns
If there are more modules than provided space, then return kernel ENOMEM.
If either handles or out_count are null, return kernel EFAULT.
* Accurate error checks in ModuleInfo functions
* Clang
* Readable VideoOutEvent data packing
Inspired by the work of former shadPS4 devs and mostly based on red_prig's current code.
* Apply DceData struct to sceVideoOutGetEventCount
Makes the code easier to read
* Update equeue.h
* Update main.cpp
* Update equeue.h
* Proper struct names
* Fix hint mask
Thanks to red_prig for catching my mistake here.
* Clang
* Fix header discrepancy
* Emulate memory behavior of libSceGnmDriver _DT_INIT
Due to the unique way some games check for sceKernelAllocateDirectMemory failures, emulating this properly is necessary.
* Clang
* Fix address input for direct memory call
* Fix bug with DirectQueryAvailable
Missed this in my prior PR.
* DirectQueryAvailable fix
Fixes error cases to be more hardware accurate.
* AllocateDirectMemory fixes
search_start and search_end were ignored in certain cases, this fixes that issue.
I've also basically rewritten the function in the process, since the lack of documentation made it difficult to make the proper adjustments.
* DirectQueryAvailable fixes
remaining_size was calculated incorrectly in cases where a free dmem_area had a base earlier than search_start, or an end after search_end.
* Reduce sceKernelGetDirectMemorySize log severity
By this point, we've confirmed that sceKernelGetDirectMemorySize is hardware-accurate. There's no reason to clog logs with this function, which games usually call before every sceKernelAllocateDirectMemory call.
* Clang
* Fix phys_addr_out
phys_addr_out should be equal to search_start in cases where search_start is greater than the dmem_area base.
* Dividing by zero is fun
Need to check for alignment when aligning things.
* Update memory.cpp
* Clang
* sceKernelDebugOutText
Some homebrew use this for logging, and these logs do show up in console klogs.
I wasn't sure where the most suitable place for this function would be, so I made a separate file for these debug functions.
* Implement kernel exit
Some homebrew I have use this exit when an error occurs. Since actually closing the emulator isn't implemented yet, I've used an unreachable message that logs the status code.
I've placed it in process.cpp for now, let me know if I should change that.
* Improved implementations for sceKernelDebugRaiseException functions
These functions take in two parameters, an error code and some other value that I have no idea what is for. If that second parameter is not zero, they return ORBIS_KERNEL_ERROR_EINVAL before any calls to mdbg_service.
These improved implementations add the early error return and a message with the error code to the unreachable.
* Add missing exports
Homebrew apps like to use these kernel exports of posix functions instead.
* Clang
* sceKernelOpen: Clean up flag handling
* sceKernelOpen: fix params
Based on decompilation, the second parameter of _open should be flags.
Additionally swaps the return and parameter types to align with our current standards.
* sceKernelOpen: Fix errors
Based on POSIX spec, if part of the path is missing, ENOENT is the correct return.
Additionally, decompilation suggests that open sets errno too.
* Fix exports
Fixes function exports to align with what they should be, based on what I've seen from decompilation and our module generator.
* Proper errno behavior on open
Since sceKernelOpen calls posix_open, errno should be set during this process.
Simplest way to handle that is to move the actual open code to posix_open and adjust error cases accordingly.
* Reorganize open calls, add error log
* Improve close
Removes the EPERM return, as it makes no sense, and swaps sceKernelClose with posix_close to properly emulate errno behavior.
* Fix log on close
* posix_open fixups
* Readd hack in posix_close
It's either this, or removing LLE DiscMap.
Or shadow implements posix sockets.
* Missing exports
Commented out while I gradually work through them all
* Remaining placeholder exports
* Swap some stuff around
I see nothing that suggests "open" only takes two parameters, so this should be completely safe.
It's also more accurate to how these are handled in libkernel, and means I won't need to reorganize anything for readv and writev.
* Update file_system.cpp
* Implement write and posix_write
* Oops
* Implement posix_readv and sceKernelReadv
Also fixes error behavior on readv, as that function shouldn't be returning any kernel error codes.
* Move sceKernelUnlink
Will deal with this one later, was just annoyed by how it's location doesn't align with the export order.
* Cleanup readv
* Implement posix_writev and sceKernelWritev
Also fixes error behavior on writev, since it shouldn't ever return kernel errors (since our device files return those)
* More cleanup on older functions
* Swap around sceKernelLseek and posix_lseek
This ensures that these have the correct error behavior, and makes their behavior align with the updated implementations for earlier functions.
* Update file_system.cpp
* Implement read
Also fixes error behavior
* Swap sceKernelMkdir and posix_mkdir
Fixes errno behavior on kernel calls, also fixed some incorrect error returns.
* Fix errno behavior on sceKernelRmdir
Also reduces function logging to bring it closer to the level of logging seen in other filesystem functions.
* Slight clean up of sceKernelStat
Fixes error behavior and changes some of the old data types.
* Refactor fstat
Fixes errno behavior, implements fstat, and shifts exports around based on file position.
Might reorganize function locations later though.
* Implement posix_ftruncate
Implements posix_ftruncate and fixes errno behavior for sceKernelFtruncate
* Add missing error conversions for more device functions
* Implement posix_rename, fix sceKernelRename errno behavior
* Add posix_preadv and posix_pread
Also fixes some incorrect error returns, fixes errno behavior, and removes an unnecessary hack.
* Fix compile
* Implement posix_getdents, getdirentries, and posix_getdirentries
Also fixes errno behavior for the kernel variants of these functions.
* Fix errno behavior of sceKernelFsync
* Implement posix_pwrite and posix_unlink
Also fixes errno behavior in related functions.
* Update file_system.cpp
* Remove SetPosixErrno
Ideally, we've handled all possible error conditions before calling these functions, so treat errors in platform-specific code as IO errors and return POSIX_EIO instead.
* Update header exports
Not sure where these get used, but might as well keep them consistent with the rest of this.
* Check if file exists before calling platform-specific code
Bloodborne checks if a file doesn't exist using open, checking if it specifically failed with error code ENOENT. To avoid working with platform-specific errnos, add a proper error return for if the file doesn't exist.
Fixes a regression in Bloodborne.
* Clang
Out of all the changes, this is apparently the only thing Clang-Format doesn't like.
I'm honestly surprised.
* Improve error checks on posix_unlink
Just because a file isn't opened doesn't mean the file doesn't exist.
Fixes the error returned if host_path.empty(), and removes the error return for when GetFile fails.
* Fix the Bloodborne fix
* Limit exports to tested functions
* More confirmed working exports
* Remaining stuff my games can test
* FS exports from firmware tests
* Bring back missing exports from main
I don't have any bootable things that call these, but since they were working well enough on main, they should be fine to readd.
* Add export for posix_pread
Spotted in Dreams a while back, might as well add it.
* Revert "Remove SetPosixErrno"
This reverts commit bdfc0c246c.
* Revert SetPosixErrno changes
shadow's using it for posix sockets, so significant modifications would introduce unnecessary merge conflicts.
* Update comment
* Add EACCES errno to SetPosixErrno
Seen in Gravity Rush.
Also reorganizes the switch case based on the posix errno value, since ordering by errno makes no sense on some OSes.
* More export fixups
Missed these during my initial pass through FS stuff because they were in kernel.cpp for some reason.
* Symbols from FS tests
Tested by messing around with firmware elfs, these atleast don't cause any crashes.
* Remove inaccurate error behavior
Seek can have offsets past the end of a file.
Also add logging for two valid whence values that are unsupported on Windows.
I'll need to verify that SEEK_HOLE and SEEK_DATA correspond to 3 and 4 respectively, I've yet to check source to verify.
* Fix error log
Oops
* Clang
Clang
* Remove close hack
Since LLE libSceDiscMap is no longer a concern, this hack shouldn't be needed.
Since sockets are still stubbed, and close can be used on sockets, I've added a warning log just in case this still occurs in some titles.
* Change SetPosixErrno unreachable to warning
I changed it to an unreachable in an earlier commit to make testing easier.
At this point, having an unreachable for this seems unnecessary, so change it to a warning instead.
* Remove Bloodborne hack
Games should be able to unlink files that aren't opened file descriptors. As far as I've tested, this doesn't break Bloodborne.
* Using OrbisKernelTimespec under clock_gettime, orbis_clock_gettime, sceKernelClockGettime to fix compatibility issues.
* final fix test
* Roamic suggestions
* host_shaders: support for includes
* video_core: add a simpler vulkan asserts
* video_core: refactored post processing pipeline to another file
* renderer_vulkan: add define param to compile shader utility
* video_core: fsr implementation
* devtools: show resolution & fsr state
* Fix error return on sceKernelMunmap
FreeBSD docs state that len <= 0 is a EINVAL return.
* Early return on ReleaseDirectMemory with len = 0
Calls to these two functions with len = 0 cause an assert in CarveDmemArea. Since there's no memory to release, an early return should be safe here.
* Remove check for negative length in munmap
Addresses review comment
Unity, being the awful game engine it is, checks for a return value of zero to determine if sceKernelLoadStartModule failed. This results in it throwing an error code into sceKernelDlsym's handle parameter when the module it's searching for doesn't exist.
* Handle errors in sceKernelAllocateDirectMemory
* Improve accuracy of error cases
Some of our existing cases are normally EAGAIN returns.
* Improve logging on errors
* Clang
* TEMPORARY HACK FOR NBA TESTS
This will be removed before this PR is marked as ready, and is only here to make sure the other NBA games (and perhaps DOA3) work if some missing init behavior is handled.
* Revert "TEMPORARY HACK FOR NBA TESTS"
This reverts commit a0e27b0229.
* Change error message
* Fix event data for VideoOut events
Fix is based on some decompilation work shared by red_prig.
* Cleanup
* Oops
* Style fixes
* Clang
* Fix libSceVideoOut event idents
Based on some decompilation work, events coming from libSceVideoOut use a separate set of values for identifiers. These values are only converted to OrbisVideoOutEventId values during calls to sceVideoOutGetEventId.
For convenience, I've placed all relevant identifiers into a enum called OrbisVideoOutInternalEventId.
Thanks to @red_prig for the tips.
* Fix?
Seems like `static_cast<u32>(hint) & 0xFF == event.ident` here, and doing those right shifts on the event.ident winds up breaking stuff.
Without this change, the if always fails because event_id was getting set to 0 instead.
* Clang
