Takes advantage of the recently implemented devoptab interfaces. Supports individual file dumping and multiple file/dir dumping via highlighting. May possibly need some additional testing. The code could potentially be improved, but I'll look into it at a later time -- I just want to release this now.
Other changes include:
* devoptab: comment out log messages.
* utils: add utilsGetDirectorySize() function.
* romfs: add hash bucket fields to RomFileSystemContext struct.
* romfs: remove current offset fields from RomFileSystemContext struct and all functions that relied on them.
* romfs: add romfsCalculateEntryHash().
* romfs: update romfsInitializeContext() to also load file/directory hash buckets.
* romfs: update romfsGetTotalDataSize() to make up for the removed functions.
* romfs: update romfsGetChild*EntryByName() functions to use hash bucket lookups.
Other changes include:
* devoptab: add missing exit macros to some FS operations across all interfaces.
* poc: update extractedRomFsReadThreadFunc() to make up for the removed RomFS functions.
This commit implements a devoptab wrapper for RomFS sections within NCAs, using code from romfs.c/h.
Other changes include:
* devoptab: add additional safety checks and some minor tweaks to both hfs_dev and pfs_dev calls.
Improve dirnext() calls in both PFS and HFS devoptab wrappers to make them also return bogus "." and ".." directory entries.
The "st_ino" field is now also being filled for all file entries.
Other changes include:
* libs: update libusbhsfs.
This commit implements a devoptab wrapper for Hash FS sections within gamecard images, using code from hfs.c/h.
Other changes include:
* poc: disable buffering on FILE objects via setvbuf().
* gamecard: fix crash in gamecardInitializeHashFileSystemContext() while calculating the size for the root Hash FS partition.
This commit implements a devoptab wrapper for Partition FS sections within NCAs, using code from pfs.c/h.
Other changes include:
* codebase: use NX_IGNORE_ARG macro where needed.
* hfs: slight tweaks to some of the static functions.
* pfs: slight tweaks to some of the static functions.
* pfs: use pfsIsValidContext() where needed.
* utils: update utilsInitializeResources() to use __system_argc and __system_argv variables from libnx.
* todo: update text file.
Other changes include:
* poc: add a "reset settings" option to the root menu.
* poc: add a message about using the controller sticks to scroll faster.
* poc: update missing ticket message to be more detailed.
* poc: add "retrieved" flag to MenuElementOption struct, which is reset while going back to a parent menu via menuResetAttributes().
* poc: fix index/count values in title selection screens.
`nsEnsureGameCardAccess()` has supposedly helped another user to fix the gamecard launch error after exiting their application, but it's not working under nxdumptool. I have yet to find out why.
Other changes include:
* host: rename windows_install_deps.py -> install_deps.py and update it to support multiple operating systems.
* host: restore assertions and fix most static analysis errors related to type aliases and checks.
* host: catch exceptions thrown by usb.core.find() in usbGetDeviceEndpoints().
* host: move code to unconditionally enable 32-bit paths under Windows to uiStartServer() and cliInitialize(), respectively.
* host: remove incomplete file if the ongoing transfer was cancelled and we're not dealing with a NSP file.
Implements both `StartExtractedFsDump` and `EndExtractedFsDump` commands into both nxdumptool and the Python host script.
Other changes include:
* poc: wake the write thread up if a preprocessing error occurs in all extracted FS dump functions. Fixes previously unhandled hangups.
* poc: verify current NCA hash before sending its last data chunk while dumping a NSP.
* host: update command handler to support CancelFileTransfer commands issued in-between SendFileProperties commands.
* host: update Markdown document.
* usb: use UsbCommandType_Count for the safety check in usbPrepareCommandHeader().
* usb: log both USB command header and command block whenever an error is reported by the host side.
libnx now implements fsDeviceOperatorGetGameCardIdSet(), so I got rid of my own implementation.
Other changes include:
* cnmt: add cnmtVerifyContentHash().
* defines: add SHA256_HASH_STR_SIZE.
* fs_ext: add FsCardId1MakerCode, FsCardId1MemoryType and FsCardId2CardType enums.
* fs_ext: update FsCardId* structs.
* gamecard: change all package_id definitions from u64 -> u8[0x8].
* gamecard: fix misleading struct member names in GameCardHeader.
* gamecard: rename gamecardGetIdSet() -> gamecardGetCardIdSet().
* gamecard_tab: fix Package ID printing.
* gamecard_tab: add Card ID Set printing.
* host: add executable flag to Python scripts.
* keys: detect if we're dealing with a wiped eTicket RSA device key (e.g. via set:cal blanking). If so, the application will still launch, but all operations related to personalized titlekey crypto are disabled.
* pfs: rename PartitionFileSystemFileContext -> PartitionFileSystemImageContext and propagate the change throughout the codebase.
* pfs: rename PFS_FULL_HEADER_ALIGNMENT -> PFS_HEADER_PADDING_ALIGNMENT and update pfsWriteImageContextHeaderToMemoryBuffer() accordingly.
* poc: print certain button prompts with reversed colors, in the hopes of getting the user's attention.
* poc: NSP, Ticket and NCA submenus for updates and DLC updates now display the highest available title by default.
* poc: simplified output path generation for extracted NCA FS section dumps.
* poc: handle edge cases where a specific NCA from an update has no matching equivalent by type/ID offset in its base title (e.g. Fall Guys' HtmlDocument NCA).
* poc: implement NCA checksum validation while generating NSP dumps.
* romfs: update romfsInitializeContext() to allow its base_nca_fs_ctx argument to be NULL.
* usb: use USB_BOS_SIZE only once.
* workflow: update commit hash referenced by "rewrite-prerelease" tag on update.
Implement utilsParseHexString() and utilsAppletLoopDelay().
Other changes include:
* defines: add THIRTY_FPS_DELAY define.
* defines: remove unused CERT_PATH define.
* gamecard: restore original reserved_2 field in GameCardInfo struct. The last 16 bytes weren't being properly due to an issue within libnx's AES-CBC implementation.
* gamecard: rename GAMECARD_ACCESS_WAIT_TIME -> GAMECARD_ACCESS_DELAY.
* host: improve ABI documentation (thanks @v1993).
* keys: remove keysConvertHexDigitToBinary().
* keys: change keysParseHexKey() function signature + update it to use utilsParseHexString() instead.
* nso: change "NRO" to "NSO" in some misleading log error messages (thanks @liamadvance).
* nxdt_rw_poc: replace svcSleepThread() calls with utilsAppletLoopDelay().
* nxdt_utils: rename utilsGenerateHexStringFromData() -> utilsGenerateHexString().
* nxdt_utils: add static utilsConvertHexDigitToBinary() function, used by utilsParseHexString().
* nxdt_utils: update utilsAppendFormattedStringToBuffer() to avoid logging an error string if resources haven't been initialized.
* nxdt_utils: use a single call to sprintf() with a variable precision argument in utilsGenerateFormattedSizeString().
* nxdt_utils: call utilsAppletLoopDelay() in the while loop from utilsPrintConsoleError().
* nxdt_utils: remove unused utilsCloseFileDescriptor() function.
* tik: log input rights ID at the start of tikRetrieveTicketByRightsId().
The host script can’t run on Python 3.9 because of [PEP 604][pep]-style annotations. Adding a future import can fix this. The `print_function` import is being removed since it is unnecessary on all releases of Python 3.
[pep]: https://peps.python.org/pep-0604/
* signature: add comments to SignatureType enum entries about the exact signing algorithms and padding schemes used.
* signature: rename signatureGetSigType() -> signatureGetTypeFromSignedBlob().
* signature: rename signatureIsValidSigType() -> signatureIsValidType().
* signature: rename signatureGetSigSize() -> signatureGetSigSizeByType().
* signature: rename signatureGetBlockSize() -> signatureGetBlockSizeByType().
* signature: rename signatureGetSig() -> signatureGetSigFromSignedBlob().
* signature: rename signatureGetPayload() -> signatureGetPayloadFromSignedBlob().
* signature: add signatureGetBlockSizeFromSignedBlob().
* cert: add more comments to the code.
* cert: update code to match signature interface changes.
* cert: add CERT_RSA_PUB_EXP_SIZE macro.
* cert: change public_exponent field in CertPublicKeyBlockRsa* structs from u32 to u8 array.
* cert: add size field to CertificateChain struct.
* cert: rename certGetCommonBlock() -> certGetCommonBlockFromSignedCertBlob.
* cert: rename certGetPublicKeySize() -> certGetPublicKeySizeByType().
* cert: rename certGetPublicKeyBlockSize() -> certGetPublicKeyBlockSizeByType().
* cert: rename certIsValidCertificate() -> certIsValidSignedCertBlob().
* cert: rename certGetSignedCertificateSize() -> certGetSignedCertBlobSize().
* cert: rename certGetSignedCertificateHashAreaSize() -> certGetSignedCertBlobHashAreaSize().
* cert: remove certGetPublicKey(), certGetPublicExponent() and certCalculateRawCertificateChainSize().
* cert: add certGetPublicKeyTypeFromCommonBlock(), certGetPublicKeyTypeFromSignedCertBlob(), certGetPublicKeySizeFromSignedCertBlob(), certGetPublicKeyBlockSizeFromSignedCertBlob(), certGetPublicKeyFromSignedCertBlob(), certGetPublicExponentFromSignedCertBlob(), certIsValidCertificate() (w/diff func sig), certGetCommonBlockFromCertificate(), certGetPublicKeyTypeFromCertificate(), certGetPublicKeySizeFromCertificate(), certGetPublicKeyBlockSizeFromCertificate(), certGetPublicKeyFromCertificate(), certGetPublicExponentFromCertificate() and certGetHashAreaSizeFromCertificate() functions.
* cert: avoid byteswapping the public key type value in multiple places -- it is now only being done in certGetPublicKeyTypeFromCommonBlock().
* cert: call certFreeCertificateChain() in _certRetrieveCertificateChainBySignatureIssuer() before attempting to retrieve the certificate chain.
* cert: other minor changes and corrections.
* tik: update code to match signature interface changes.
* tik: add missing comments to TikPropertyMask enum entries.
* tik: add key_generation, enc_titlekey_str and dec_titlekey_str fields to Ticket struct.
* tik: update tikRetrieveTicketByRightsId() to also take in a key_generation argument, instead of getting it from the rights ID (which could fail if it's using a key generation lower than HOS 3.0.1) or the key_generation field from the common ticket block (which could fail if the ticket has been tampered by certain tools).
* tik: rename tikGetCommonBlock() -> tikGetCommonBlockFromSignedTicketBlob().
* tik: change function signature for tikGetTicketSectionRecordsBlockSize().
* tik: rename tikIsValidTicket() -> tikIsValidSignedTicketBlob().
* tik: rename tikGetSignedTicketSize() -> tikGetSignedTicketBlobSize().
* tik: rename tikGetSignedTicketHashAreaSize() -> tikGetSignedTicketBlobHashAreaSize().
* tik: rename tikGetEncryptedTitleKeyFromTicket() -> tikGetEncryptedTitleKey().
* tik: add tikIsValidTicket() (w/diff func sig), tikGetCommonBlockFromTicket(), tikGetHashAreaSizeFromTicket(), tikFixTamperedCommonTicket(), tikVerifyRsa2048Sha256Signature() and tikDecryptVolatileTicket() functions. Ticket signature verification is only carried out for common tickets in tikFixTamperedCommonTicket().
* tik: change argument order in tikGetTicketEntryOffsetFromTicketList() and tikRetrieveTicketEntryFromTicketBin().
* tik: add TIK_COMMON_CERT_NAME and TIK_DEV_CERT_ISSUER macros.
* tik: use a scoped lock when calling tikRetrieveTicketFromEsSaveDataByRightsId().
* tik: simplify certificate chain retrieval steps in tikConvertPersonalizedTicketToCommonTicket() by always using the XS00000020 certificate.
* tik: wipe license_type and property_mask fields in tikConvertPersonalizedTicketToCommonTicket().
* tik: other minor changes and corrections.
Other changes include:
* keys: fix key generation checks in keysGetNcaKeyAreaKeyEncryptionKey() and keysGetTicketCommonKey().
* rsa: move core logic from rsa2048VerifySha256BasedPssSignature() into a new function: rsa2048VerifySha256BasedSignature().
* rsa: add rsa2048VerifySha256BasedPkcs1v15Signature() function.
ncaStorageInitializeContext() now takes an extra input argument. ncaStorageSetPatchOriginalSubStorage() is now a static function that's only used internally.
Fixes issues with "AKIBA'S TRIP: Undead & Undressed Director's Cut" compressed storage initialization while using an update. Big thanks to @Arch9SK7 for reporting the issue.
system_title_dumper PoC build is no more.
Other changes include:
* smc: add missing assert for SmcGenerateAesKekOption struct.
* title: update titleGenerateFileName() to add support for system titles.
Affects both utilsAppendFormattedStringToBuffer() and logWriteFormattedStringToBuffer(). Fixes logging issues within both the exception handler and memory debugging code.
Other changes include:
* bfttf: rename BfttfFontType_Total -> BfttfFontType_Count.
* config: rename "append_authoringtool_data" -> "generate_authoringtool_data".
* fs_ext: update FsGameCardCertificate struct
* gamecard: fix gamecardReadLotusAsicFirmwareBlob() not returning false if FS .data segment memory couldn't be retrieved; update GameCardInfo struct to reflect a recently discovered area that's not zeroed out.
* mem: expand MemoryProgramSegmentType enum; define extra macros for PID buffer size and memory page type checks; force empty memory page attribute and R/X permission checks while looking for the last FS .text segment; make memory page filtering code more readable.
* npdm: rename KernelCapability enums; rename entry_value -> bitmask in all KernelCapability descriptor structs.
* tik: force byte-for-byte memory lookup while dumping volatile tickets.
* libs: update libusbhsfs to latest commit.
* codebase: add missing comments to some enums; add missing "None" and "Count" elements to some enums. "Count" entries from enums with bitmasks will only reflect the number of available bitmask values. Enums with hex values remain unchanged.
* PoC builds: use EXIT_SUCCESS and EXIT_FAILURE as return values.
* nxdt_rw_poc: move "end" jump label within main() to avoid a crash if utilsInitializeResources() fails; reflect changes in config.c.
Pablo Curiel
Johnson
bilditup1
TSRBerry
Andy Dirnberger
Vinfall