diff --git a/.gitignore b/.gitignore index db044b5ad..fd402a2a6 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,4 @@ cmake-* .cache .ccls compile_commands.json +releases/ diff --git a/README.md b/README.md index 73fa960c1..848dac304 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,13 @@ MeshCore is a lightweight, portable C++ library that enables multi-hop packet routing for embedded projects using LoRa and other packet radios. It is designed for developers who want to create resilient, decentralized communication networks that work without the internet. +## About the Dogechat Bridge + +This is a special experimental Dogechat bridge branch. + +Read [my blog](https://juraj.bednar.io/en/blog-en/2026/01/18/bridging-dogechat-and-meshcore-resilient-communication-when-you-need-it-most/) and [documentation](docs/dogechat-bridge.md). + + ## 🔍 What is MeshCore? MeshCore now supports a range of LoRa devices, allowing for easy flashing without the need to compile firmware manually. Users can flash a pre-built binary using tools like Adafruit ESPTool and interact with the network through a serial console. @@ -97,10 +104,10 @@ Here are some general principals you should try to adhere to: There are a number of fairly major features in the pipeline, with no particular time-frames attached yet. In very rough chronological order: - [X] Companion radio: UI redesign -- [ ] Repeater + Room Server: add ACL's (like Sensor Node has) -- [ ] Standardise Bridge mode for repeaters +- [X] Repeater + Room Server: add ACL's (like Sensor Node has) +- [X] Standardise Bridge mode for repeaters - [ ] Repeater/Bridge: Standardise the Transport Codes for zoning/filtering -- [ ] Core + Repeater: enhanced zero-hop neighbour discovery +- [X] Core + Repeater: enhanced zero-hop neighbour discovery - [ ] Core: round-trip manual path support - [ ] Companion + Apps: support for multiple sub-meshes (and 'off-grid' client repeat mode) - [ ] Core + Apps: support for LZW message compression @@ -113,12 +120,3 @@ There are a number of fairly major features in the pipeline, with no particular - Report bugs and request features on the [GitHub Issues](https://github.com/ripplebiz/MeshCore/issues) page. - Find additional guides and components on [my site](https://buymeacoffee.com/ripplebiz). - Join [MeshCore Discord](https://discord.gg/BMwCtwHj5V) to chat with the developers and get help from the community. - -## RAK Wireless Board Support in PlatformIO - -Before building/flashing the RAK4631 targets in this project, there is, unfortunately, some patching you have to do to your platformIO packages to make it work. There is a guide here on the process: - [RAK Wireless: How to Perform Installation of Board Support Package in PlatformIO](https://learn.rakwireless.com/hc/en-us/articles/26687276346775-How-To-Perform-Installation-of-Board-Support-Package-in-PlatformIO) - -After building, you will need to convert the output firmware.hex file into a .uf2 file you can copy over to your RAK4631 device (after doing a full erase) by using the command `uf2conv.py -f 0xADA52840 -c firmware.hex` with the python script available from: - [GitHub: Microsoft - uf2](https://github.com/Microsoft/uf2/blob/master/utils/uf2conv.py) - diff --git a/docs/dogechat-bridge.md b/docs/dogechat-bridge.md new file mode 100644 index 000000000..168f8949a --- /dev/null +++ b/docs/dogechat-bridge.md @@ -0,0 +1,148 @@ +# Dogechat Bridge + +The Dogechat Bridge enables communication between the [Dogechat](https://dogechat.app) Android app and MeshCore mesh network. Messages sent to the `#mesh` channel in Dogechat are relayed to MeshCore nodes, and vice versa. + +## Overview + +- **Bridge Direction**: Bidirectional - messages flow both ways between Dogechat and MeshCore +- **Channel**: Only the `#mesh` channel is bridged (hardcoded) +- **Identification**: Messages from Dogechat users appear with a phone emoji prefix on MeshCore nodes +- **Platform**: ESP32 only (requires ESP32 ROM miniz for decompression) + +## Supported Boards + +The following ESP32 boards have `_companion_radio_usb_dogechat` targets: + +| Board | Target Name | +|-------|-------------| +| Heltec LoRa32 V2 | `Heltec_v2_companion_radio_usb_dogechat` | +| Heltec LoRa32 V3 | `Heltec_v3_companion_radio_usb_dogechat` | +| Heltec Wireless Stick Lite V3 | `Heltec_WSL3_companion_radio_usb_dogechat` | +| Heltec CT62 | `Heltec_ct62_companion_radio_usb_dogechat` | +| Heltec Tracker V2 | `heltec_tracker_v2_companion_radio_usb_dogechat` | +| LilyGo T3-S3 (SX1262) | `LilyGo_T3S3_sx1262_companion_radio_usb_dogechat` | +| LilyGo T-Deck | `LilyGo_TDeck_companion_radio_usb_dogechat` | +| LilyGo TLora V2.1 | `LilyGo_TLora_V2_1_1_6_companion_radio_usb_dogechat` | +| Station G2 | `Station_G2_companion_radio_usb_dogechat` | +| Seeed Xiao C3 | `Xiao_C3_companion_radio_usb_dogechat` | +| Seeed Xiao S3 WIO | `Xiao_S3_WIO_companion_radio_usb_dogechat` | +| Ebyte EoRa-S3 | `Ebyte_EoRa-S3_companion_radio_usb_dogechat` | + +**Note**: NRF52-based boards are not supported because Dogechat message decompression requires the ESP32 ROM miniz library. + +## Build Targets + +### USB + Dogechat (`*_companion_radio_usb_dogechat`) + +This is the recommended configuration: + +- **MeshCore companion app**: Connects via USB serial +- **Dogechat app**: Connects via standalone BLE + +This gives the best experience because: +- The MeshCore web app works reliably via USB serial +- Dogechat has dedicated BLE access without sharing with MeshCore + +### Building + +```bash +# Build for Heltec Wireless Stick Lite V3 +pio run -e Heltec_WSL3_companion_radio_usb_dogechat + +# Flash +pio run -e Heltec_WSL3_companion_radio_usb_dogechat -t upload +``` + +## How It Works + +### Channel Bridging + +1. **Dogechat to MeshCore**: When a Dogechat user sends a message to `#mesh`, the bridge: + - Receives the message via BLE + - Decompresses it (Dogechat uses zlib compression) + - Extracts the sender nickname and message content + - Sends it to the MeshCore #mesh with a phone emoji prefix + +2. **MeshCore to Dogechat**: When a MeshCore node sends a message to the `#mesh` channel, the bridge: + - Receives the mesh packet + - Creates a Dogechat MESSAGE packet with the sender name and content + - Broadcasts it via BLE + +### Message Format + +Messages from Dogechat appear on MeshCore as: +``` +: message content +``` + +Messages from MeshCore appear on Dogechat with the MeshCore sender's name in angle brackets. + +### Long Messages + +MeshCore packets have a ~127-byte payload limit. Long Dogechat messages are automatically split into multiple parts with `[1/N]` indicators. + +## Limitations + +1. **Only #mesh channel**: Only the `#mesh` hashtag channel is bridged. DMs and other channels are ignored. + +2. **Message size**: MeshCore packets are limited to ~127 bytes. Long messages are split into multiple parts. + +3. **No end-to-end encryption bridge**: Dogechat's Noise protocol encryption and MeshCore's encryption are separate. Messages are decrypted/re-encrypted at the bridge. + +4. **No file/image transfer**: Dogechat file transfers (images, etc.) are not supported on the mesh. + +5. **Time synchronization**: The bridge synchronizes its clock from incoming Dogechat packets. If no Dogechat client connects, timestamps may be inaccurate. Time needs to be accurate for Dogechat to work + +## Debugging + +To enable debug output, use one of these methods: + +### Method 1: Environment variable (temporary) + +```bash +# Build with debug output enabled +PLATFORMIO_BUILD_FLAGS="-D DOGECHAT_DEBUG=1" pio run -e Heltec_WSL3_companion_radio_usb_dogechat +``` + +### Method 2: Edit platformio.ini (persistent) + +Add `-D DOGECHAT_DEBUG=1` to your environment's build_flags: + +```ini +[env:Heltec_WSL3_companion_radio_usb_dogechat] +build_flags = + ${Heltec_WSL3_companion_radio_usb.build_flags} + ${dogechat_base.build_flags} + -D DOGECHAT_DEBUG=1 ; Enable debug output +``` + +This enables detailed logging of: +- Message parsing +- Relay confirmations +- Peer cache updates +- Fragment reassembly + +## Technical Details + +### BLE Service + +- **Service UUID**: `F47B5E2D-4A9E-4C5A-9B3F-8E1D2C3A4B5C` +- **Characteristic**: Single read/write/notify characteristic +- **MTU**: 517 bytes (max BLE MTU for large messages) + +### Channel Key + +The `#mesh` channel uses a hashtag-derived key: +``` +SHA256("#mesh")[0:16] = 5b664cde0b08b220612113db980650f3 +``` + +This matches MeshCore's hashtag room key derivation. + +### Protocol Support + +- Message decompression (zlib via ESP32 ROM miniz) +- Ed25519 message signing +- Peer announcement/discovery +- REQUEST_SYNC handling (sends cached messages) +- Fragment reassembly for long messages diff --git a/docs/faq.md b/docs/faq.md index efed5a864..66a942a4a 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -26,6 +26,10 @@ author: https://github.com/LitBomb - [3.2. Q: Do I need to set the location for a repeater?](#32-q-do-i-need-to-set-the-location-for-a-repeater) - [3.3. Q: What is the password to administer a repeater or a room server?](#33-q-what-is-the-password-to-administer-a-repeater-or-a-room-server) - [3.4. Q: What is the password to join a room server?](#34-q-what-is-the-password-to-join-a-room-server) + - [3.5. Q: Can I retrieve a repeater's private key or set a repeater's private key?](#35-q-can-i-retrieve-a-repeaters-private-key-or-set-a-repeaters-private-key) + - [3.6. Q: The first byte of my repeater's public key collides with an exisitng repeater on the mesh. How do I get a new private key with a matching public key that has its first byte of my choosing?](#36-q-the-first-byte-of-my-repeaters-public-key-collides-with-an-exisitng-repeater-on-the-mesh--how-do-i-get-a-new-private-key-with-a-matching-public-key-that-has-its-first-byte-of-my-choosing) + - [3.7. Q: My repeater maybe suffering from deafness due to high power interference near my mesh's frequency, it is not hearing other in-range MeshCore radios. what can I do?](#37-q-my-repeater-maybe-suffering-from-deafness-due-to-high-power-interference-near-my-meshs-frequency-it-is-not-hearing-other-in-range-meshcore-radios--what-can-i-do) + - [3.8 Q: How do I make my repeater an observer on the mesh](#38-q-how-do-i-make-my-repeater-an-observer-on-the-mesh) - [4. T-Deck Related](#4-t-deck-related) - [4.1. Q: Is there a user guide for T-Deck, T-Pager, T-Watch, or T-Display Pro?](#41-q-is-there-a-user-guide-for-t-deck-t-pager-t-watch-or-t-display-pro) - [4.2. Q: What are the steps to get a T-Deck into DFU (Device Firmware Update) mode?](#42-q-what-are-the-steps-to-get-a-t-deck-into-dfu-device-firmware-update-mode) @@ -61,22 +65,31 @@ author: https://github.com/LitBomb - [5.14.3. Python MeshCore](#5143-python-meshcore) - [5.14.4. meshcore-cli](#5144-meshcore-cli) - [5.14.5. meshcore.js](#5145-meshcorejs) + - [5.14.6. pyMC\_core](#5146-pymc_core) + - [5.14.7. MeshCore Packet Decoder](#5147-meshcore-packet-decoder) + - [5.14.8. meshcore-pi](#5148-meshcore-pi) + - [5.14.9. pyMC\_Repeater](#5149-pymc_repeater) + - [5.15. Q: Are there client applications for Windows or Mac?](#515-q-are-there-client-applications-for-windows-or-mac) + - [5.16. Q: Are there any resources that compare MeshCore to other LoRa systems?](#516-q-are-there-any-resources-that-compare-meshcore-to-other-lora-systems) - [6. Troubleshooting](#6-troubleshooting) - [6.1. Q: My client says another client or a repeater or a room server was last seen many, many days ago.](#61-q-my-client-says-another-client-or-a-repeater-or-a-room-server-was-last-seen-many-many-days-ago) - [6.2. Q: A repeater or a client or a room server I expect to see on my discover list (on T-Deck) or contact list (on a smart device client) are not listed.](#62-q-a-repeater-or-a-client-or-a-room-server-i-expect-to-see-on-my-discover-list-on-t-deck-or-contact-list-on-a-smart-device-client-are-not-listed) - [6.3. Q: How to connect to a repeater via BLE (Bluetooth)?](#63-q-how-to-connect-to-a-repeater-via-ble-bluetooth) - [6.4. Q: My companion isn't showing up over Bluetooth?](#64-q-my-companion-isnt-showing-up-over-bluetooth) - - [6.5. Q: I can't connect via Bluetooth, what is the Bluetooth pairing code?](#64-q-i-cant-connect-via-bluetooth-what-is-the-bluetooth-pairing-code) - - [6.6. Q: My Heltec V3 keeps disconnecting from my smartphone. It can't hold a solid Bluetooth connection.](#65-q-my-heltec-v3-keeps-disconnecting-from-my-smartphone--it-cant-hold-a-solid-bluetooth-connection) - - [6.7. Q: My RAK/T1000-E/xiao\_nRF52 device seems to be corrupted, how do I wipe it clean to start fresh?](#66-q-my-rakt1000-exiao_nrf52-device-seems-to-be-corrupted-how-do-i-wipe-it-clean-to-start-fresh) - - [6.8. Q: WebFlasher fails on Linux with failed to open](#67-q-webflasher-fails-on-linux-with-failed-to-open) + - [6.5. Q: I can't connect via Bluetooth, what is the Bluetooth pairing code?](#65-q-i-cant-connect-via-bluetooth-what-is-the-bluetooth-pairing-code) + - [6.6. Q: My Heltec V3 keeps disconnecting from my smartphone. It can't hold a solid Bluetooth connection.](#66-q-my-heltec-v3-keeps-disconnecting-from-my-smartphone--it-cant-hold-a-solid-bluetooth-connection) + - [6.7. Q: My RAK/T1000-E/xiao\_nRF52 device seems to be corrupted, how do I wipe it clean to start fresh?](#67-q-my-rakt1000-exiao_nrf52-device-seems-to-be-corrupted-how-do-i-wipe-it-clean-to-start-fresh) + - [6.8. Q: WebFlasher fails on Linux with failed to open](#68-q-webflasher-fails-on-linux-with-failed-to-open) - [7. Other Questions:](#7-other-questions) - [7.1. Q: How to update nRF (RAK, T114, Seed XIAO) repeater and room server firmware over the air using the new simpler DFU app?](#71-q-how-to-update-nrf-rak-t114-seed-xiao-repeater-and-room-server-firmware-over-the-air-using-the-new-simpler-dfu-app) + - [7.1.1 Q: Can I update Seeed Studio Wio Tracker L1 Pro using OTA?](#711-q-can-i-update-seeed-studio-wio-tracker-l1-pro-using-ota) - [7.2. Q: How to update ESP32-based devices over the air?](#72-q-how-to-update-esp32-based-devices-over-the-air) - [7.3. Q: Is there a way to lower the chance of a failed OTA device firmware update (DFU)?](#73-q-is-there-a-way-to-lower-the-chance-of-a-failed-ota-device-firmware-update-dfu) - [7.4. Q: are the MeshCore logo and font available?](#74-q-are-the-meshcore-logo-and-font-available) - [7.5. Q: What is the format of a contact or channel QR code?](#75-q-what-is-the-format-of-a-contact-or-channel-qr-code) - - [7.6. Q: How do I connect to the companion via WIFI, e.g. using a heltec v3?](#76-q-how-do-i-connect-to-the-comnpanion-via-wifi-eg-using-a-heltec-v3) + - [7.6. Q: How do I connect to the companion via WIFI, e.g. using a heltec v3?](#76-q-how-do-i-connect-to-the-companion-via-wifi-eg-using-a-heltec-v3) + - [7.7. Q: I have a Station G2, or a Heltec V4, or an Ikoka Stick, or a radio with a EByte E22-900M30S or a E22-900M33S module, what should their transmit power be set to?](#77-q-i-have-a-station-g2-or-a-heltec-v4-or-an-ikoka-stick-or-a-radio-with-a-ebyte-e22-900m30s-or-a-e22-900m33s-module-what-should-their-transmit-power-be-set-to) +- [| | High Output | 22 dBm | 28 dBm | |](#--high-output--22-dbm--28-dbm--) ## 1. Introduction @@ -91,7 +104,7 @@ MeshCore is free and open source: * The T-Deck firmware is developed by Scott at Ripple Radios, the creator of MeshCore, is also free to flash on your devices and use -Some more advanced, but optional features are available on T-Deck if you register your device for a key to unlock. On the MeshCore smartphone clients for Android and iOS/iPadOS, you can unlock the wait timer for repeater and room server remote management over RF feature. +Some more advanced, but optional features are available on T-Deck if you register your device for a key to unlock. On the MeshCore smartphone clients for Android and iOS/iPadOS, you can unlock the wait timer for repeater and room server remote management over RF feature. These features are completely optional and aren't needed for the core messaging experience. They're like super bonus features and to help the developers continue to work on these amazing features, they may charge a small fee for an unlock code to utilise the advanced features. @@ -105,7 +118,7 @@ Anyone is able to build anything they like on top of MeshCore without paying any MeshCore Firmware GitHub: https://github.com/ripplebiz/MeshCore NOTE: Andy Kirby has a very useful [intro video](https://www.youtube.com/watch?v=t1qne8uJBAc) for beginners. - + You need LoRa hardware devices to run MeshCore firmware as clients or server (repeater and room server). @@ -114,7 +127,7 @@ MeshCore is available on a variety of 433MHz, 868MHz and 915MHz LoRa devices. Fo For an up-to-date list of supported devices, please go to https://flasher.meshcore.co.uk/ -To use MeshCore without using a phone as the client interface, you can run MeshCore on a LiLygo's T-Deck, T-Deck Plus, T-Pager, T-Watch, or T-Display Pro. MeshCore Ultra firmware running on these devices are a complete off-grid secure communication solution. +To use MeshCore without using a phone as the client interface, you can run MeshCore on a LiLygo's T-Deck, T-Deck Plus, T-Pager, T-Watch, or T-Display Pro. MeshCore Ultra firmware running on these devices are a complete off-grid secure communication solution. #### 1.2.2. Firmware MeshCore has four firmware types that are not available on other LoRa systems. MeshCore has the following: @@ -122,30 +135,30 @@ MeshCore has four firmware types that are not available on other LoRa systems. M #### 1.2.3. Companion Radio Firmware Companion radios are for connecting to the Android app or web app as a messenger client. There are two different companion radio firmware versions: -1. **BLE Companion** - BLE Companion firmware runs on a supported LoRa device and connects to a smart device running the Android or iOS MeshCore client over BLE +1. **BLE Companion** + BLE Companion firmware runs on a supported LoRa device and connects to a smart device running the Android or iOS MeshCore client over BLE -2. **USB Serial Companion** - USB Serial Companion firmware runs on a supported LoRa device and connects to a smart device or a computer over USB Serial running the MeshCore web client - +2. **USB Serial Companion** + USB Serial Companion firmware runs on a supported LoRa device and connects to a smart device or a computer over USB Serial running the MeshCore web client + #### 1.2.4. Repeater -Repeaters are used to extend the range of a MeshCore network. Repeater firmware runs on the same devices that run client firmware. A repeater's job is to forward MeshCore packets to the destination device. It does **not** forward or retransmit every packet it receives, unlike other LoRa mesh systems. +Repeaters are used to extend the range of a MeshCore network. Repeater firmware runs on the same devices that run client firmware. A repeater's job is to forward MeshCore packets to the destination device. It does **not** forward or retransmit every packet it receives, unlike other LoRa mesh systems. A repeater can be remotely administered using a T-Deck running the MeshCore firmware with remote administration features unlocked, or from a BLE Companion client connected to a smartphone running the MeshCore app. #### 1.2.5. Room Server -A room server is a simple BBS server for sharing posts. T-Deck devices running MeshCore firmware or a BLE Companion client connected to a smartphone running the MeshCore app can connect to a room server. +A room server is a simple BBS server for sharing posts. T-Deck devices running MeshCore firmware or a BLE Companion client connected to a smartphone running the MeshCore app can connect to a room server. Room servers store message history on them and push the stored messages to users. Room servers allow roaming users to come back later and retrieve message history. With channels, messages are either received when it's sent, or not received and missed if the channel user is out of range. Room servers are different and more like email servers where you can come back later and get your emails from your mail server. -A room server can be remotely administered using a T-Deck running the MeshCore firmware with remote administration features unlocked, or from a BLE Companion client connected to a smartphone running the MeshCore app. +A room server can be remotely administered using a T-Deck running the MeshCore firmware with remote administration features unlocked, or from a BLE Companion client connected to a smartphone running the MeshCore app. When a client logs into a room server, the client will receive the previously 32 unseen messages. -Although room server can also repeat with the command line command `set repeat on`, it is not recommended nor encouraged. A room server with repeat set to `on` lacks the full set of repeater and remote administration features that are only available in the repeater firmware. +Although room server can also repeat with the command line command `set repeat on`, it is not recommended nor encouraged. A room server with repeat set to `on` lacks the full set of repeater and remote administration features that are only available in the repeater firmware. The recommendation is to run repeater and room server on separate devices for the best experience. @@ -168,37 +181,32 @@ After you flashed the latest firmware onto your repeater device, keep the device The repeater and room server CLI reference is here: https://github.com/meshcore-dev/MeshCore/wiki/Repeater-&-Room-Server-CLI-Reference -If you have more supported devices, you can use your additional devices with the room server firmware. +If you have more supported devices, you can use your additional devices with the room server firmware. ### 2.2. Q: Does MeshCore cost any money? -**A:** All radio firmware versions (e.g. for Heltec V3, RAK, T-1000E, etc) are free and open source developed by Scott at Ripple Radios. +**A:** All radio firmware versions (e.g. for Heltec V3, RAK, T-1000E, etc) are free and open source developed by Scott at Ripple Radios. -The native Android and iOS client uses the freemium model and is developed by Liam Cottle, developer of meshtastic map at [meshtastic.liamcottle.net](https://meshtastic.liamcottle.net) on [GitHub](https://github.com/liamcottle/meshtastic-map) and [reticulum-meshchat on github](https://github.com/liamcottle/reticulum-meshchat). +The native Android and iOS client uses the freemium model and is developed by Liam Cottle, developer of meshtastic map at [meshtastic.liamcottle.net](https://meshtastic.liamcottle.net) on [GitHub](https://github.com/liamcottle/meshtastic-map) and [reticulum-meshchat on github](https://github.com/liamcottle/reticulum-meshchat). -The T-Deck firmware is free to download and most features are available without cost. To support the firmware developer, you can pay for a registration key to unlock your T-Deck for deeper map zoom and remote server administration over RF using the T-Deck. You do not need to pay for the registration to use your T-Deck for direct messaging and connecting to repeaters and room servers. +The T-Deck firmware is free to download and most features are available without cost. To support the firmware developer, you can pay for a registration key to unlock your T-Deck for deeper map zoom and remote server administration over RF using the T-Deck. You do not need to pay for the registration to use your T-Deck for direct messaging and connecting to repeaters and room servers. ### 2.3. Q: What frequencies are supported by MeshCore? -**A:** It supports the 868MHz range in the UK/EU and the 915MHz range in New Zealand, Australia, and the USA. Countries and regions in these two frequency ranges are also supported. The firmware and client allow users to set their preferred frequency. -- Australia and New Zealand are on **915.8MHz** -- UK and EU are on **869.525MHz** -- Canada and USA are on **910.525MHz** -- For other regions and countries, please check your local LoRa frequency +**A:** It supports the 868MHz range in the UK/EU and the 915MHz range in New Zealand, Australia, and the USA. Countries and regions in these two frequency ranges are also supported. + +Use the smartphone client or the repeater setup feature on there web flasher to set your radios' RF settings by choosing the preset for your regions. -In UK and EU, 867.5MHz is not allowed to use 250kHz bandwidth and it only allows 2.5% duty cycle for clients. 869.525Mhz allows an airtime of 10%, 250KHz bandwidth, and a higher EIRP, therefore MeshCore nodes can send more often and with more power. That is why this frequency is chosen for UK and EU. This is also why Meshtastic also uses this frequency. +Recently, as of October 2025, many regions have moved to the "narrow" setting, aka using BW62.5 and a lower SF number (instead of the original SF11). For example, USA/Canada (Recommended) preset is 910.525MHz, SF7, BW62.5, CR5. -[Source](https://discord.com/channels/826570251612323860/1330643963501351004/1356540643853209641) +After extensive testing, many regions have switched or about to switch over to BW62.5 and SF7, 8, or 9. Narrower bandwidth setting and lower SF setting allow MeshCore's radio signals to fit between interference in the ISM band, provide for a lower noise floor, better SNR, and faster transmissions. + +If you have consensus from your community in your region to update your region's preset recommendation, please post your update request on the [#meshcore-app](https://discord.com/channels/1343693475589263471/1391681655911088241) channel on the [MeshCore Discord server ](https://discord.gg/cYtQNYCCRK) to let Liam Cottle know. -the rest of the radio settings are the same for all frequencies: -- Spread Factor (SF): 11 -- Coding Rate (CR): 5 -- Bandwidth (BW): 250.00 -(Originally MeshCore started with SF 10. recently (as of late April 2025) the community has advocated SF 11 also a viable option for longer range but a little slower transmission. Currently there are MeshCore meshes with SF 10 and SF 11. Liam Cottle's smartphone app's presets now recommend SF 10 for Australia and SF 11 for all other regions and countries. EU and UK has SF 10 and SF 11 presets. Work with your local meshers on deciding with SF number is best for your use cases. In the future, there may be bridge nodes that can bridge SF 10 and SF 11 (or even different frequencies) traffic.) ### 2.4. Q: What is an "advert" in MeshCore? -**A:** +**A:** Advert means to advertise yourself on the network. In Reticulum terms it would be to announce. In Meshtastic terms it would be the node sending its node info. MeshCore allows you to manually broadcast your name, position and public encryption key, which is also signed to prevent spoofing. When you click the advert button, it broadcasts that data over LoRa. MeshCore calls that an Advert. There's two ways to advert, "zero hop" and "flood". @@ -214,7 +222,7 @@ As of Aug 20 2025, a pending PR on github will change the flood advert to 12 hou ### 2.5. Q: Is there a hop limit? -**A:** Internally the firmware has maximum limit of 64 hops. In real world settings it will be difficult to get close to the limit due to the environments and timing as packets travel further and further. We want to hear how far your MeshCore conversations go. +**A:** Internally the firmware has maximum limit of 64 hops. In real world settings it will be difficult to get close to the limit due to the environments and timing as packets travel further and further. We want to hear how far your MeshCore conversations go. --- @@ -224,14 +232,14 @@ As of Aug 20 2025, a pending PR on github will change the flood advert to 12 hou ### 3.1. Q: How do you configure a repeater or a room server? -**A:** - When MeshCore is flashed onto a LoRa device is for the first time, it is necessary to set the server device's frequency to make it utilize the frequency that is legal in your country or region. +**A:** - When MeshCore is flashed onto a LoRa device is for the first time, it is necessary to set the server device's frequency to make it utilize the frequency that is legal in your country or region. Repeater or room server can be administered with one of the options below: - + - After a repeater or room server firmware is flashed on to a LoRa device, go to and use the web user interface to connect to the LoRa device via USB serial. From there you can set the name of the server, its frequency and other related settings, location, passwords etc. ![image](https://github.com/user-attachments/assets/2a9d9894-e34d-4dbe-b57c-fc3c250a2d34) - + - Connect the server device using a USB cable to a computer running Chrome on https://flasher.meshcore.co.uk/, then use the `console` feature to connect to the device - Use a MeshCore smartphone clients to remotely administer servers via LoRa. @@ -240,10 +248,10 @@ Repeater or room server can be administered with one of the options below: - + ### 3.2. Q: Do I need to set the location for a repeater? -**A:** With location set for a repeater, it can show up on a MeshCore map in the future. Set location with the following commands: +**A:** While not required, with location set for a repeater it will show up on the MeshCore map in the future. Set location with the following command: `set lat set long ` @@ -260,6 +268,34 @@ You can get the latitude and longitude from Google Maps by right-clicking the lo `set guest.password {guest-password}` +### 3.5. Q: Can I retrieve a repeater's private key or set a repeater's private key? + +**A:** You can issue these commands to get or set a repeater's private key using a USB serial connection. + +`get prv.key` to print a repeater's private key on the serial console +`set prv.key ` to set a repeater's private key on the serial console + +Reboot the repeater after `set prv.key ` command for the new private key to take effect. + +### 3.6. Q: The first byte of my repeater's public key collides with an exisitng repeater on the mesh. How do I get a new private key with a matching public key that has its first byte of my choosing? + +**A:** You can generate a new private key and specific the first byte of its public key here: https://gessaman.com/mc-keygen/ + + +### 3.7. Q: My repeater maybe suffering from deafness due to high power interference near my mesh's frequency, it is not hearing other in-range MeshCore radios. what can I do? + +**A:** This may be due to the SX1262 radio's auto gain control feature. You can use this command to preiodically reset its AGC. + +`set agc.reset.interval ` + +The `` unit is in seconds and is incremented by 4. `set agc.reset.interval 4` works well to cure deafness. + +This is a very low cost operation. AGC reset is done by simply setting `state = STATE_IDLE;` in function `RadioLibWrapper::resetAGC()` in `RadioLibWrappers.cpp` + + +### 3.8 Q: How do I make my repeater an observer on the mesh + +**A:** The observer instruction is available here: https://analyzer.letsme.sh/observer/onboard --- @@ -270,14 +306,14 @@ You can get the latitude and longitude from Google Maps by right-clicking the lo **A:** Yes, it is available on https://buymeacoffee.com/ripplebiz/ultra-v7-7-guide-meshcore-users ### 4.2. Q: What are the steps to get a T-Deck into DFU (Device Firmware Update) mode? -**A:** -1. Device off -2. Connect USB cable to device -3. Hold down trackball (keep holding) -4. Turn on device -5. Hear USB connection sound -6. Release trackball -7. T-Deck in DFU mode now +**A:** +1. Device off +2. Connect USB cable to device +3. Hold down trackball (keep holding) +4. Turn on device +5. Hear USB connection sound +6. Release trackball +7. T-Deck in DFU mode now 8. At this point you can begin flashing using ### 4.3. Q: Why is my T-Deck Plus not getting any satellite lock? @@ -294,10 +330,12 @@ GPS on T-Deck is always enabled. You can skip the "GPS clock sync" and the T-De **A:** Users have had no issues using 16GB or 32GB SD cards. Format the SD card to **FAT32**. ### 4.6. Q: what is the public key for the default public channel? -**A:** -T-Deck uses the same key the smartphone apps use but in base64 +**A:** +T-Deck uses the same key the smartphone apps use but in base64 `izOH6cXN6mrJ5e26oRXNcg==` -The third character is the capital letter 'O', not zero `0` + +There is no `=` key on the T-Deck's hardware keyboard. You can use the on-screen software keyboard to enter `=`. Tap the text box to enable the on-screen software keyboard. +The third character is the capital letter `O` (Oh), not zero `0` The smartphone app key is in hex: ` 8b3387e9c5cdea6ac9e5edbaa115cd72` @@ -305,24 +343,24 @@ The smartphone app key is in hex: [Source](https://discord.com/channels/826570251612323860/1330643963501351004/1354194409213792388) ### 4.7. Q: How do I get maps on T-Deck? -**A:** You need map tiles. You can get pre-downloaded map tiles here (a good way to support development): -- (Europe) +**A:** You need map tiles. You can get pre-downloaded map tiles here (a good way to support development): +- (Europe) - (US) -Another way to download map tiles is to use this Python script to get the tiles in the areas you want: - +Another way to download map tiles is to use this Python script to get the tiles in the areas you want: + -There is also a modified script that adds additional error handling and parallel downloads: - +There is also a modified script that adds additional error handling and parallel downloads: + -UK map tiles are available separately from Andy Kirby on his discord server: +UK map tiles are available separately from Andy Kirby on his discord server: ### 4.8. Q: Where do the map tiles go? Once you have the tiles downloaded, copy the `\tiles` folder to the root of your T-Deck's SD card. ### 4.9. Q: How to unlock deeper map zoom and server management features on T-Deck? -**A:** You can download, install, and use the T-Deck firmware for free, but it has some features (map zoom, server administration) that are enabled if you purchase an unlock code for \$10 per T-Deck device. +**A:** You can download, install, and use the T-Deck firmware for free, but it has some features (map zoom, server administration) that are enabled if you purchase an unlock code for \$10 per T-Deck device. Unlock page: ### 4.10. Q: How to decipher the diagnostics screen on T-Deck? @@ -330,17 +368,17 @@ Unlock page: **A: ** Space is tight on T-Deck's screen, so the information is a bit cryptic. The format is : `{hops} l:{packet-length}({payload-len}) t:{packet-type} snr:{n} rssi:{n}` -See here for packet-type: +See here for packet-type: https://github.com/meshcore-dev/MeshCore/blob/main/src/Packet.h#L19 - - - #define PAYLOAD_TYPE_REQ 0x00 // request (prefixed with dest/src hashes, MAC) (enc data: timestamp, blob) - #define PAYLOAD_TYPE_RESPONSE 0x01 // response to REQ or ANON_REQ (prefixed with dest/src hashes, MAC) (enc data: timestamp, blob) - #define PAYLOAD_TYPE_TXT_MSG 0x02 // a plain text message (prefixed with dest/src hashes, MAC) (enc data: timestamp, text) - #define PAYLOAD_TYPE_ACK 0x03 // a simple ack #define PAYLOAD_TYPE_ADVERT 0x04 // a node advertising its Identity - #define PAYLOAD_TYPE_GRP_TXT 0x05 // an (unverified) group text message (prefixed with channel hash, MAC) (enc data: timestamp, "name: msg") - #define PAYLOAD_TYPE_GRP_DATA 0x06 // an (unverified) group datagram (prefixed with channel hash, MAC) (enc data: timestamp, blob) - #define PAYLOAD_TYPE_ANON_REQ 0x07 // generic request (prefixed with dest_hash, ephemeral pub_key, MAC) (enc data: ...) + + + #define PAYLOAD_TYPE_REQ 0x00 // request (prefixed with dest/src hashes, MAC) (enc data: timestamp, blob) + #define PAYLOAD_TYPE_RESPONSE 0x01 // response to REQ or ANON_REQ (prefixed with dest/src hashes, MAC) (enc data: timestamp, blob) + #define PAYLOAD_TYPE_TXT_MSG 0x02 // a plain text message (prefixed with dest/src hashes, MAC) (enc data: timestamp, text) + #define PAYLOAD_TYPE_ACK 0x03 // a simple ack #define PAYLOAD_TYPE_ADVERT 0x04 // a node advertising its Identity + #define PAYLOAD_TYPE_GRP_TXT 0x05 // an (unverified) group text message (prefixed with channel hash, MAC) (enc data: timestamp, "name: msg") + #define PAYLOAD_TYPE_GRP_DATA 0x06 // an (unverified) group datagram (prefixed with channel hash, MAC) (enc data: timestamp, blob) + #define PAYLOAD_TYPE_ANON_REQ 0x07 // generic request (prefixed with dest_hash, ephemeral pub_key, MAC) (enc data: ...) #define PAYLOAD_TYPE_PATH 0x08 // returned path (prefixed with dest/src hashes, MAC) (enc data: path, extra) [Source](https://discord.com/channels/1343693475589263471/1343693475589263474/1350611321040932966) @@ -370,14 +408,30 @@ https://github.com/meshcore-dev/MeshCore/blob/main/src/Packet.h#L19 ### 5.1. Q: What are BW, SF, and CR? -**A:** +**A:** **BW is bandwidth** - width of frequency spectrum that is used for transmission **SF is spreading factor** - how much should the communication spread in time -**CR is coding rate** - https://www.thethingsnetwork.org/docs/lorawan/fec-and-code-rate/ -Making the bandwidth 2x wider (from BW125 to BW250) allows you to send 2x more bytes in the same time. Making the spreading factor 1 step lower (from SF10 to SF9) allows you to send 2x more bytes in the same time. +**CR is coding rate** - from: https://www.thethingsnetwork.org/docs/lorawan/fec-and-code-rate/ + +TL;DR: default CR to 5 for good stable links. If it is not a solid link and is intermittent, change to CR to 7 or 8. + +Forward Error Correction is a process of adding redundant bits to the data to be transmitted. During the transmission, data may get corrupted by interference (changes from 0 to 1 / 1 to 0). These error correction bits are used at the receivers for restoring corrupted bits. + +The Code Rate of a forward error correction expresses the proportion of bits in a data stream that actually carry useful information. + +There are 4 code rates used in LoRaWAN: + +4/5 +4/6 +5/7 +4/8 + +For example, if the code rate is 5/7, for every 5 bits of useful information, the coder generates a total of 7 bits of data, of which 2 bits are redundant. + +Making the bandwidth 2x wider (from BW125 to BW250) allows you to send 2x more bytes in the same time. Making the spreading factor 1 step lower (from SF10 to SF9) allows you to send 2x more bytes in the same time. Lowering the spreading factor makes it more difficult for the gateway to receive a transmission, as it will be more sensitive to noise. You could compare this to two people taking in a noisy place (a bar for example). If you’re far from each other, you have to talk slow (SF10), but if you’re close, you can talk faster (SF7) @@ -385,14 +439,14 @@ So, it's balancing act between speed of the transmission and resistance to noise things network is mainly focused on LoRaWAN, but the LoRa low-level stuff still checks out for any LoRa project ### 5.2. Q: Do MeshCore clients repeat? -**A:** No, MeshCore clients do not repeat. This is the core of MeshCore's messaging-first design. This is to avoid devices flooding the air ware and create endless collisions, so messages sent aren't received. -In MeshCore, only repeaters and room server with `set repeat on` repeat. +**A:** No, MeshCore clients do not repeat. This is the core of MeshCore's messaging-first design. This is to avoid devices flooding the air ware and create endless collisions, so messages sent aren't received. +In MeshCore, only repeaters and room server with `set repeat on` repeat. ### 5.3. Q: What happens when a node learns a route via a mobile repeater, and that repeater is gone? **A:** If you used to reach a node through a repeater and the repeater is no longer reachable, the client will send the message using the existing (but now broken) known path, the message will fail after 3 retries, and the app will reset the path and send the message as flood on the last retry by default. This can be turned off in settings. If the destination is reachable directly or through another repeater, the new path will be used going forward. Or you can set the path manually if you know a specific repeater to use to reach that destination. -In the case if users are moving around frequently, and the paths are breaking, they just see the phone client retries and revert to flood to attempt to re-establish a path. +In the case if users are moving around frequently, and the paths are breaking, they just see the phone client retries and revert to flood to attempt to re-establish a path. ### 5.4. Q: How does a node discovery a path to its destination and then use it to send messages in the future, instead of flooding every message it sends like Meshtastic? @@ -411,14 +465,14 @@ Routes are stored in sender's contact list. When you send a message the first t **A:** The smartphone app key is in hex: ` 8b3387e9c5cdea6ac9e5edbaa115cd72` -T-Deck uses the same key but in base64 +T-Deck uses the same key but in base64 `izOH6cXN6mrJ5e26oRXNcg==` The third character is the capital letter 'O', not zero `0` [Source](https://discord.com/channels/826570251612323860/1330643963501351004/1354194409213792388) ### 5.7. Q: Is MeshCore open source? -**A:** Most of the firmware is freely available. Everything is open source except the T-Deck firmware and Liam's native mobile apps. -- Firmware repo: https://github.com/meshcore-dev/MeshCore +**A:** Most of the firmware is freely available. Everything is open source except the T-Deck firmware and Liam's native mobile apps. +- Firmware repo: https://github.com/meshcore-dev/MeshCore ### 5.8. Q: How can I support MeshCore? **A:** Provide your honest feedback on GitHub and on [MeshCore Discord server](https://discord.gg/BMwCtwHj5V). Spread the word of MeshCore to your friends and communities; help them get started with MeshCore. Support Scott's MeshCore development at . @@ -428,7 +482,7 @@ Support Liam Cottle's smartphone client development by unlocking the server admi Support Rastislav Vysoky (recrof)'s flasher web site and the map web site development through [PayPal](https://www.paypal.com/donate/?business=DREHF5HM265ES&no_recurring=0&item_name=If+you+enjoy+my+work%2C+you+can+support+me+here%3A¤cy_code=EUR) or [Revolut](https://revolut.me/recrof) ### 5.9. Q: How do I build MeshCore firmware from source? -**A:** See instructions here: +**A:** See instructions here: https://discord.com/channels/826570251612323860/1330643963501351004/1341826372120608769 Build instructions for MeshCore: @@ -448,7 +502,7 @@ Then it should be the same for all platforms: python3 -m venv meshcore cd meshcore && source bin/activate pip install -U platformio -git clone https://github.com/ripplebiz/MeshCore.git +git clone https://github.com/ripplebiz/MeshCore.git cd MeshCore ``` open platformio.ini and in `[arduino_base]` edit the `LORA_FREQ=867.5` @@ -458,8 +512,8 @@ pio run -e RAK_4631_Repeater ``` then you'll find `firmware.zip` in `.pio/build/RAK_4631_Repeater` -Andy also has a video on how to build using VS Code: -*How to build and flash Meshcore repeater firmware | Heltec V3* +Andy also has a video on how to build using VS Code: +*How to build and flash Meshcore repeater firmware | Heltec V3* *(Link referenced in the Discord post)* ### 5.10. Q: Are there other MeshCore related open source projects? @@ -476,13 +530,13 @@ Meshcore would not be best suited to ATAK because MeshCore: clients do not repeat and therefore you would need a network of repeaters in place will not have a stable path where all clients are constantly moving between repeaters -MeshCore clients would need to reset path constantly and flood traffic across the network which could lead to lots of collisions with something as chatty as ATAK. +MeshCore clients would need to reset path constantly and flood traffic across the network which could lead to lots of collisions with something as chatty as ATAK. This could change in the future if MeshCore develops a client firmware that repeats. [Source](https://discord.com/channels/826570251612323860/1330643963501351004/1354780032140054659) ### 5.12. Q: How do I add a node to the [MeshCore Map]([url](https://meshcore.co.uk/map.html)) -**A:** +**A:** To add a BLE Companion radio, connect to the BLE Companion radio from the MeshCore smartphone app. In the app, tap the `3 dot` menu icon at the top right corner, then tap `Internet Map`. Tap the `3 dot` menu icon again and choose `Add me to the Map` @@ -501,7 +555,7 @@ For ESP-based devices (e.g. Heltec V3) you need: - Download firmware file from flasher.meshcore.co.uk - Go to the web site on a browser, find the section that has the firmware up need - Click the Download button, right click on the file you need, for example, - - `Heltec_V3_companion_radio_ble-v1.7.1-165fb33.bin` + - `Heltec_V3_companion_radio_ble-v1.7.1-165fb33.bin` - Non-merged bin keeps the existing Bluetooth pairing database - `Heltec_v3_companion_radio_usb-v1.7.1-165fb33-merged.bin` - Merged bin overwrites everything including the bootloader, existing Bluetooth pairing database, but keeps configurations. @@ -520,7 +574,7 @@ For ESP-based devices (e.g. Heltec V3) you need: - `esptool.py -p /dev/ttyUSB0 --chip esp32-s3 write_flash 0x10000 .bin` - For merged bin: - `esptool.py -p /dev/ttyUSB0 --chip esp32-s3 write_flash 0x00000 .bin` - + **Instructions for nRF devices:** @@ -541,24 +595,25 @@ For nRF devices (e.g. RAK, Heltec T114) you need the following: - `pip install adafruit-nrfutil --break-system-packages` - Use this command to flash the nRF device: - `adafruit-nrfutil --verbose dfu serial --package RAK_4631_companion_radio_usb-v1.7.1-165fb33.zip -p /dev/ttyACM0 -b 115200 --singlebank --touch 1200` - - + + To manage a repeater or room server connected to a Pi over USB serial using shell commands, you need to install `picocom`. To install `picocom`, run the following command: - `sudo apt install picocom` To start managing your USB serial-connected device using picocom, use the following command: - `picocom -b 115200 /dev/ttyUSB0 --imap lfcrlf` -From here, reference repeater and room server command line commands on MeshCore github wiki here: +From here, reference repeater and room server command line commands on MeshCore github wiki here: - https://github.com/meshcore-dev/MeshCore/wiki/Repeater-&-Room-Server-CLI-Reference - + ### 5.14. Q: Are there are projects built around MeshCore? **A:** Yes. See the following: #### 5.14.1. meshcoremqtt -A Python script to send meshore debug and packet capture data to MQTT for analysis +A Python script to send meshcore debug and packet capture data to MQTT for analysis. Cisien's version is a fork of Andrew-a-g's and is being used to to collect data for https://map.w0z.is/messages and https://analyzer.letsme.sh/ +https://github.com/Cisien/meshcoretomqtt https://github.com/Andrew-a-g/meshcoretomqtt #### 5.14.2. MeshCore for Home Assistant @@ -569,7 +624,7 @@ https://github.com/awolden/meshcore-ha Bindings to access your MeshCore companion radio nodes in python. https://github.com/fdlamotte/meshcore_py -#### 5.14.4. meshcore-cli +#### 5.14.4. meshcore-cli CLI interface to MeshCore companion radio over BLE, TCP, or serial. Uses Python MeshCore above. https://github.com/fdlamotte/meshcore-cli @@ -577,15 +632,49 @@ CLI interface to MeshCore companion radio over BLE, TCP, or serial. Uses Python A JavaScript library for interacting with a MeshCore device running the companion radio firmware https://github.com/liamcottle/meshcore.js +#### 5.14.6. pyMC_core +pyMC_Core is a Python port of MeshCore, designed for Raspberry Pi and similar hardware, it talks to LoRa modules over SPI. +https://github.com/rightup/pyMC_core + +#### 5.14.7. MeshCore Packet Decoder +A TypeScript library for decoding MeshCore mesh networking packets with full cryptographic support. Uses WebAssembly (WASM) for Ed25519 key derivation through the orlp/ed25519 library. It powers the [MeshCore Packet Analyzer](https://analyzer.letsme.sh/packets). +https://github.com/michaelhart/meshcore-decoder + +#### 5.14.8. meshcore-pi +meshcore-pi is another Python port of MeshCore, designed for Raspberry Pi and similar hardware, it talks to LoRa modules over SPI or GPIO. +https://github.com/brianwiddas/meshcore-pi + +#### 5.14.9. pyMC_Repeater +pyMC_Repeater is a repeater daemon in Python built on top of the [`pymc_core`](#5146-pymc_core) library. +https://github.com/rightup/pyMC_Repeater + + +### 5.15. Q: Are there client applications for Windows or Mac? +**A:** Yes, the same iOS and Android client is also available for Windows and Intel Mac (sorry, not available for ARM-based Mac yet). You can find them together with the Android APK here: +https://files.liamcottle.net/MeshCore + +Both the Windows and Intel Mac versions of the client app are fully unlocked and are free to use. + +### 5.16. Q: Are there any resources that compare MeshCore to other LoRa systems? + +**A:** Here is a list of MeshCore comparison resources: +The Comms Channel on YouTube: +https://www.youtube.com/watch?v=guDoKGs02Us +MeshCore Advantages by MCarper: +https://github.com/mikecarper/meshfirmware/blob/main/MeshCoreAdvantages.md +Meshcore vs Meshtastic by austinmesh.org +https://www.austinmesh.org/learn/meshcore-vs-meshtastic/ + + --- ## 6. Troubleshooting ### 6.1. Q: My client says another client or a repeater or a room server was last seen many, many days ago. ### 6.2. Q: A repeater or a client or a room server I expect to see on my discover list (on T-Deck) or contact list (on a smart device client) are not listed. -**A:** -- If your client is a T-Deck, it may not have its time set (no GPS installed, no GPS lock, or wrong GPS baud rate). -- If you are using the Android or iOS client, the other client, repeater, or room server may have the wrong time. +**A:** +- If your client is a T-Deck, it may not have its time set (no GPS installed, no GPS lock, or wrong GPS baud rate). +- If you are using the Android or iOS client, the other client, repeater, or room server may have the wrong time. You can get the epoch time on and use it to set your T-Deck clock. For a repeater and room server, the admin can use a T-Deck to remotely set their clock (clock sync), or use the `time` command in the USB serial console with the server device connected. @@ -606,23 +695,23 @@ You can get the epoch time on and use it to se ### 6.7. Q: My RAK/T1000-E/xiao_nRF52 device seems to be corrupted, how do I wipe it clean to start fresh? -**A:** +**A:** 1. Connect USB-C cable to your device, per your device's instruction, get it to flash mode: - For RAK, click the reset button **TWICE** - For T1000-e, quickly disconnect and reconnect the magnetic side of the cable from the device **TWICE** - For Heltec T114, click the reset button **TWICE** (the bottom button) - For Xiao nRF52, click the reset button once. If that doesn't work, quickly double click the reset button twice. If that doesn't work, disconnection the board from your PC and reconnect again ([seeed studio wiki](https://wiki.seeedstudio.com/XIAO_BLE/#access-the-swd-pins-for-debugging-and-reflashing-bootloader)) 5. A new folder will appear on your computer's desktop -6. Download the `flash_erase*.uf2` file for your device on flasher.meshcore.co.uk +6. Download the `flash_erase*.uf2` file for your device on flasher.meshcore.co.uk - RAK WisBlock and Heltec T114: `Flash_erase-nRF32_softdevice_v6.uf2` - Seeed Studio Xiao nRF52 WIO: `Flash_erase-nRF52_softdevice_v7.uf2` 8. drag and drop the uf2 file for your device to the root of the new folder 9. Wait for the copy to complete. You might get an error dialog, you can ignore it -10. Go to https://flasher.meshcore.co.uk/, click `Console` and select the serial port for your connected device +10. Go to https://flasher.meshcore.co.uk/, click `Console` and select the serial port for your connected device 11. In the console, press enter. Your flash should now be erased 12. You may now flash the latest MeshCore firmware onto your device -Separately, starting in firmware version 1.7.0, there is a CLI Rescue mode. If your device has a user button (e.g. some RAK, T114), you can activate the rescue mode by hold down the user button of the device within 8 seconds of boot. Then you can use the 'Console' on flasher.meshcore.co.uk +Separately, starting in firmware version 1.7.0, there is a CLI Rescue mode. If your device has a user button (e.g. some RAK, T114), you can activate the rescue mode by hold down the user button of the device within 8 seconds of boot. Then you can use the 'Console' on flasher.meshcore.co.uk ### 6.8. Q: WebFlasher fails on Linux with failed to open @@ -645,14 +734,20 @@ Allow the browser user on it: 4. Go to the Command Line tab, type `start ota` and hit enter. 5. you should see `OK` to confirm the repeater device is now in OTA mode 6. Run the DFU app,tab `Settings` on the top right corner -7. Enable `Packets receipt notifications`, and change `Number of Packets` to 10 for RAK, 8 for T114. 8 also works for RAK. +7. Enable `Packets receipt notifications`, and change `Number of Packets` to 10 for RAK, 8 for T114. 8 also works for RAK. 9. Select the firmware zip file you downloaded 10. Select the device you want to update. If the device you want to update is not on the list, try enabling`OTA` on the device again 11. If the device is not found, enable `Force Scanning` in the DFU app 12. Tab the `Upload` to begin OTA update -13. If it fails, try turning off and on Bluetooth on your phone. If that doesn't work, try rebooting your phone. +13. If it fails, try turning off and on Bluetooth on your phone. If that doesn't work, try rebooting your phone. 14. Wait for the update to complete. It can take a few minutes. +#### 7.1.1 Q: Can I update Seeed Studio Wio Tracker L1 Pro using OTA? +**A:** You can flash this safer bootloader to the Wio Tracker L1 Pro +https://github.com/oltaco/Adafruit_nRF52_Bootloader_OTAFIX + +After this bootloader is flashed onto the device, you can trigger over the air update using bluetooth by holding the button next to the D-Pad and then click the reset button. The follow the same OTA update instructions above. You can skip pass the `start ota` instruction and start the update using the DFU app. + ### 7.2. Q: How to update ESP32-based devices over the air? @@ -662,25 +757,29 @@ Allow the browser user on it: 4. Go to the Command Line tab, type `start ota` and hit enter. 5. you should see `OK` to confirm the repeater device is now in OTA mode 6. The command `start ota` on an ESP32-based device starts a wifi hotspot named `MeshCore OTA` -7. From your phone or computer connect to the 'MeshCore OTA' hotspot +7. From your phone or computer connect to the 'MeshCore OTA' hotspot 8. From a browser, go to http://192.168.4.1/update and upload the non-merged bin from the flasher ### 7.3. Q: Is there a way to lower the chance of a failed OTA device firmware update (DFU)? -**A:** Yes, developer `che aporeps` has an enhanced OTA DFU bootloader for nRF52 based devices. With this bootloader, if it detects that the application firmware is invalid, it falls back to OTA DFU mode so you can attempt to flash again to recover. This bootloader has other changes to make the OTA DFU process more fault tolerant. +**A:** Yes, developer `che aporeps` has an enhanced OTA DFU bootloader for nRF52 based devices. With this bootloader, if it detects that the application firmware is invalid, it falls back to OTA DFU mode so you can attempt to flash again to recover. This bootloader has other changes to make the OTA DFU process more fault tolerant. Refer to https://github.com/oltaco/Adafruit_nRF52_Bootloader_OTAFIX for the latest information. Currently, the following boards are supported: -- Nologo ProMicro +- Heltec Automation Mesh Node T114 / HT-nRF5262 +- Nologo ProMicro NRF52840 (aka SuperMini NRF52840) +- Seeed Studio SenseCAP Card Tracker T1000-E +- Seeed Studio Wio Tracker L1 - Seeed Studio XIAO nRF52840 BLE - Seeed Studio XIAO nRF52840 BLE SENSE -- RAK 4631 +- RAK 4631 (See note) +- RAK WisMesh Tag (new 28/11/2025) ### 7.4. Q: are the MeshCore logo and font available? -**A:** Yes, it is on the MeshCore github repo here: +**A:** Yes, it is on the MeshCore github repo here: https://github.com/meshcore-dev/MeshCore/tree/main/logo ### 7.5. Q: What is the format of a contact or channel QR code? @@ -699,8 +798,26 @@ where `&type` is: `sensor = 4` ### 7.6. Q: How do I connect to the companion via WIFI, e.g. using a heltec v3? - **A:** + **A:** WiFi firmware requires you to compile it yourself, as you need to set the wifi ssid and password. Edit WIFI_SSID and WIFI_PWD in `./variants/heltec_v3/platformio.ini` and then flash it to your device. +### 7.7. Q: I have a Station G2, or a Heltec V4, or an Ikoka Stick, or a radio with a EByte E22-900M30S or a E22-900M33S module, what should their transmit power be set to? + **A:** +For companion radios, you can set these radios' transmit power in the smartphone app. For repeater and room server radios, you can set their transmit power using the command line command `set tx`. You can get their current value using command line comand `get tx` + + +> ### ⚠️ **WARNING: Set these values at your own risk. Incorrect power settings can permanently damage your radio hardware.** + +| Device / Model | Region / Description | In-App Setting (dBm) | Target Radio Output | Notes | +| :--- | :--- | :--- | :--- | :--- | +| **Station G2**
[Reference](https://wiki.uniteng.com/en/meshtastic/station-g2) | US915 Max Output | 19 dBm | 36.5 dBm (4.46W) | | +| | US915 Recommended Max | 16 dBm | 35 dBm (3.16W) | 1dB compression point | +| | EU868 Recommended Max | 15 dBm | 34.5 dBm (2.82W) | 1dB compression point | +| | US915 1W Output | 10 dBm | 1W | | +| | EU868 1W Output | 9 dBm | 1W | | +| **Ikoka Stick E22-900M30S** | 1W Model | 19 dBm | 1W | **DO NOT EXCEED** (Risk of burn out) | +| **Ikoka Stick E22-900M33S** | 2W Model | 9 dBm | 2W | **DO NOT EXCEED** (Risk of burn out) | +| **Heltec V4** | Standard Output | 10 dBm | 22 dBm | | +| | High Output | 22 dBm | 28 dBm | | --- diff --git a/docs/payloads.md b/docs/payloads.md index 5a41e69ce..f86a70bc4 100644 --- a/docs/payloads.md +++ b/docs/payloads.md @@ -218,4 +218,4 @@ The plaintext contained in the ciphertext matches the format described in [plain # Custom packet -Custom packets have no defined format. \ No newline at end of file +Custom packets have no defined format. diff --git a/examples/companion_radio/MyMesh.cpp b/examples/companion_radio/MyMesh.cpp index 09d866c9e..5d0455dfa 100644 --- a/examples/companion_radio/MyMesh.cpp +++ b/examples/companion_radio/MyMesh.cpp @@ -3,6 +3,10 @@ #include // needed for PlatformIO #include +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) +#include +#endif + #define CMD_APP_START 1 #define CMD_SEND_TXT_MSG 2 #define CMD_SEND_CHANNEL_TXT_MSG 3 @@ -439,6 +443,34 @@ void MyMesh::onSignedMessageRecv(const ContactInfo &from, mesh::Packet *pkt, uin void MyMesh::onChannelMessageRecv(const mesh::GroupChannel &channel, mesh::Packet *pkt, uint32_t timestamp, const char *text) { +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + Serial.println("MYMESH: onChannelMessageRecv called"); + Serial.print("MYMESH: _dogechatBridge="); + Serial.println(_dogechatBridge != nullptr ? "valid" : "NULL"); + + // Forward to Dogechat bridge if available + if (_dogechatBridge != nullptr) { + // Text format is "sender: message" - extract sender and message + const char* colonPos = strchr(text, ':'); + if (colonPos != nullptr && colonPos > text) { + char senderName[32]; + size_t senderLen = colonPos - text; + if (senderLen >= sizeof(senderName)) senderLen = sizeof(senderName) - 1; + memcpy(senderName, text, senderLen); + senderName[senderLen] = '\0'; + + const char* msgText = colonPos + 1; + while (*msgText == ' ') msgText++; // Skip leading space + + _dogechatBridge->onMeshcoreGroupMessage(channel, timestamp, senderName, msgText); + } else { + // No colon found, use whole text + _dogechatBridge->onMeshcoreGroupMessage(channel, timestamp, "Unknown", text); + } + Serial.println("MYMESH: dogechat bridge call returned"); + } +#endif + int i = 0; if (app_target_ver >= 3) { out_frame[i++] = RESP_CODE_CHANNEL_MSG_RECV_V3; @@ -482,6 +514,7 @@ void MyMesh::onChannelMessageRecv(const mesh::GroupChannel &channel, mesh::Packe } if (_ui) _ui->newMsg(path_len, channel_name, text, offline_queue_len); #endif + Serial.println("MYMESH: onChannelMessageRecv COMPLETE"); } uint8_t MyMesh::onContactRequest(const ContactInfo &contact, uint32_t sender_timestamp, const uint8_t *data, @@ -730,6 +763,10 @@ MyMesh::MyMesh(mesh::Radio &radio, mesh::RNG &rng, mesh::RTCClock &rtc, SimpleMe memset(advert_paths, 0, sizeof(advert_paths)); memset(send_scope.key, 0, sizeof(send_scope.key)); +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + _dogechatBridge = NULL; +#endif + // defaults memset(&_prefs, 0, sizeof(_prefs)); _prefs.airtime_factor = 1.0; // one half @@ -1833,6 +1870,13 @@ void MyMesh::checkSerialInterface() { } void MyMesh::loop() { + // Call dogechat loop FIRST - BaseChatMesh::loop() may block on serial/radio +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + if (_dogechatBridge != nullptr) { + _dogechatBridge->loop(); + } +#endif + BaseChatMesh::loop(); if (_cli_rescue) { @@ -1865,4 +1909,10 @@ bool MyMesh::advert() { } else { return false; } -} \ No newline at end of file +} + +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) +void MyMesh::initDogechat(DogechatBridge* bridge) { + _dogechatBridge = bridge; +} +#endif \ No newline at end of file diff --git a/examples/companion_radio/MyMesh.h b/examples/companion_radio/MyMesh.h index 1fcc5697d..51ba460fe 100644 --- a/examples/companion_radio/MyMesh.h +++ b/examples/companion_radio/MyMesh.h @@ -70,6 +70,10 @@ #include #include +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) +class DogechatBridge; // Forward declaration +#endif + /* -------------------------------------------------------------------------------------- */ #define REQ_TYPE_GET_STATUS 0x01 // same as _GET_STATS @@ -95,6 +99,11 @@ class MyMesh : public BaseChatMesh, public DataStoreHost { NodePrefs *getNodePrefs(); uint32_t getBLEPin(); +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + void initDogechat(DogechatBridge* bridge); + DogechatBridge* getDogechatBridge() { return _dogechatBridge; } +#endif + void loop(); void handleCmdFrame(size_t len); bool advert(); @@ -224,6 +233,10 @@ class MyMesh : public BaseChatMesh, public DataStoreHost { #define ADVERT_PATH_TABLE_SIZE 16 AdvertPath advert_paths[ADVERT_PATH_TABLE_SIZE]; // circular table + +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + DogechatBridge* _dogechatBridge; +#endif }; extern MyMesh the_mesh; diff --git a/examples/companion_radio/main.cpp b/examples/companion_radio/main.cpp index 82c8c21d9..b4186e71a 100644 --- a/examples/companion_radio/main.cpp +++ b/examples/companion_radio/main.cpp @@ -91,6 +91,10 @@ static uint32_t _atoi(const char* sp) { UITask ui_task(&board, &serial_interface); #endif +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + #include +#endif + StdRNG fast_rng; SimpleMeshTables tables; MyMesh the_mesh(radio_driver, fast_rng, rtc_clock, tables, store @@ -99,6 +103,10 @@ MyMesh the_mesh(radio_driver, fast_rng, rtc_clock, tables, store #endif ); +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + DogechatBridge* dogechat_bridge = nullptr; +#endif + /* END GLOBAL OBJECTS */ void halt() { @@ -155,7 +163,20 @@ void setup() { sprintf(dev_name, "%s%s", BLE_NAME_PREFIX, the_mesh.getNodeName()); serial_interface.begin(dev_name, the_mesh.getBLEPin()); #else + // USB Serial for MeshCore companion serial_interface.begin(Serial); + + // Standalone Dogechat BLE if enabled (USB serial for MeshCore, BLE for Dogechat) + #ifdef ENABLE_DOGECHAT + dogechat_bridge = new DogechatBridge(the_mesh, the_mesh.self_id, the_mesh.getNodeName()); + dogechat_bridge->begin(); + if (dogechat_bridge->beginStandalone(the_mesh.getNodeName())) { + Serial.println("Dogechat BLE service started (standalone mode)"); + } else { + Serial.println("ERROR: Failed to start Dogechat BLE service!"); + } + the_mesh.initDogechat(dogechat_bridge); + #endif #endif the_mesh.startInterface(serial_interface); #elif defined(RP2040_PLATFORM) @@ -202,12 +223,39 @@ void setup() { char dev_name[32+16]; sprintf(dev_name, "%s%s", BLE_NAME_PREFIX, the_mesh.getNodeName()); serial_interface.begin(dev_name, the_mesh.getBLEPin()); + + // Initialize Dogechat bridge after BLE server is created + #ifdef ENABLE_DOGECHAT + dogechat_bridge = new DogechatBridge(the_mesh, the_mesh.self_id, the_mesh.getNodeName()); + dogechat_bridge->begin(); + if (serial_interface.getBLEServer() != nullptr) { + if (dogechat_bridge->attachBLEService(serial_interface.getBLEServer())) { + serial_interface.setDogechatService(&dogechat_bridge->getBLEService()); + Serial.println("Dogechat BLE service attached"); + } + } + the_mesh.initDogechat(dogechat_bridge); + #endif #elif defined(SERIAL_RX) companion_serial.setPins(SERIAL_RX, SERIAL_TX); companion_serial.begin(115200); serial_interface.begin(companion_serial); #else + // Default: USB Serial for MeshCore companion serial_interface.begin(Serial); + + // Standalone Dogechat BLE if enabled (no SerialBLEInterface) + #ifdef ENABLE_DOGECHAT + dogechat_bridge = new DogechatBridge(the_mesh, the_mesh.self_id, the_mesh.getNodeName()); + dogechat_bridge->begin(); + + if (dogechat_bridge->beginStandalone(the_mesh.getNodeName())) { + Serial.println("Dogechat BLE service started (standalone mode)"); + } else { + Serial.println("ERROR: Failed to start Dogechat BLE service!"); + } + the_mesh.initDogechat(dogechat_bridge); + #endif #endif the_mesh.startInterface(serial_interface); #else @@ -228,4 +276,10 @@ void loop() { ui_task.loop(); #endif rtc_clock.tick(); + +#if defined(ENABLE_DOGECHAT) && (defined(ESP32) || defined(NRF52_PLATFORM)) + if (dogechat_bridge != nullptr) { + dogechat_bridge->loop(); + } +#endif } diff --git a/examples/simple_room_server/MyMesh.cpp b/examples/simple_room_server/MyMesh.cpp index 60dd18407..d18a802e8 100644 --- a/examples/simple_room_server/MyMesh.cpp +++ b/examples/simple_room_server/MyMesh.cpp @@ -815,7 +815,7 @@ void MyMesh::loop() { if (c->extra.room.pending_ack && millisHasNowPassed(c->extra.room.ack_timeout)) { c->extra.room.push_failures++; c->extra.room.pending_ack = 0; // reset (TODO: keep prev expected_ack's in a list, incase they arrive LATER, after we retry) - MESH_DEBUG_PRINTLN("pending ACK timed out: push_failures: %d", (uint32_t)c->push_failures); + MESH_DEBUG_PRINTLN("pending ACK timed out: push_failures: %d", (uint32_t)c->extra.room.push_failures); } } // check next Round-Robin client, and sync next new post diff --git a/platformio.ini b/platformio.ini index 3907cf64b..7e4411664 100644 --- a/platformio.ini +++ b/platformio.ini @@ -66,6 +66,15 @@ lib_deps = me-no-dev/ESPAsyncWebServer @ ^3.6.0 file://arch/esp32/AsyncElegantOTA +; ----------------- Dogechat Support --------------------- +; Extend this to add Dogechat protocol bridge support +; Note: Uses ESP32 ROM miniz for decompression (no external library needed) +[dogechat_base] +build_flags = + -D ENABLE_DOGECHAT=1 +build_src_filter = + + + ; esp32c6 uses arduino framework 3.x ; WARNING: experimental. pioarduino on esp32c6 needs work - it's not considered stable and has issues. [esp32c6_base] @@ -87,6 +96,18 @@ build_flags = ${arduino_base.build_flags} lib_deps = ${arduino_base.lib_deps} https://github.com/oltaco/CustomLFS @ 0.2.1 + +; ----------------- NRF52 Dogechat Support --------------------- +; Extend this to add Dogechat protocol bridge support for NRF52 +; Uses portable miniz library for decompression +[dogechat_nrf52] +build_flags = + -D ENABLE_DOGECHAT=1 +build_src_filter = + + + + + + + + ; ----------------- RP2040 --------------------- [rp2040_base] diff --git a/src/helpers/dogechat/DogechatBLEService.cpp b/src/helpers/dogechat/DogechatBLEService.cpp new file mode 100644 index 000000000..2c1c52994 --- /dev/null +++ b/src/helpers/dogechat/DogechatBLEService.cpp @@ -0,0 +1,376 @@ +#include "DogechatBLEService.h" + +#ifdef ESP32 + +#include + +#if DOGECHAT_DEBUG + #define DOGECHAT_DEBUG_PRINTLN(F, ...) Serial.printf("DOGECHAT: " F "\n", ##__VA_ARGS__) +#else + #define DOGECHAT_DEBUG_PRINTLN(...) {} +#endif + +// Verbose packet hex dump macro (separate from DOGECHAT_DEBUG for optional verbosity) +#if DOGECHAT_DEBUG_PACKETDUMP +static void dumpPacketHex(const char* label, const uint8_t* data, size_t len) { + Serial.printf("PACKETDUMP [%s] (%zu bytes):\n", label, len); + for (size_t i = 0; i < len; i++) { + Serial.printf("%02X ", data[i]); + if ((i + 1) % 16 == 0) Serial.println(); + } + if (len % 16 != 0) Serial.println(); +} + #define DOGECHAT_PACKETDUMP(label, data, len) dumpPacketHex(label, data, len) +#else + #define DOGECHAT_PACKETDUMP(label, data, len) {} +#endif + +DogechatBLEService::DogechatBLEService() + : _server(nullptr) + , _service(nullptr) + , _characteristic(nullptr) + , _callback(nullptr) + , _serviceActive(false) + , _dogechatClientCount(0) + , _lastKnownServerCount(0) + , _clientSubscribed(false) + , _pendingConnect(false) + , _pendingData(false) + , _writeBufferOffset(0) + , _lastWriteTime(0) + , _queueHead(0) + , _queueTail(0) +{ + memset(_writeBuffer, 0, sizeof(_writeBuffer)); + memset(_deviceName, 0, sizeof(_deviceName)); + strcpy(_deviceName, "Dogechat"); + for (size_t i = 0; i < MESSAGE_QUEUE_SIZE; i++) { + _messageQueue[i].valid = false; + } +} + +bool DogechatBLEService::attachToServer(BLEServer* server, DogechatBLECallback* callback) { + if (server == nullptr || callback == nullptr) { + DOGECHAT_DEBUG_PRINTLN("attachToServer: null server or callback"); + return false; + } + + _server = server; + _callback = callback; + + // Create Dogechat service + _service = _server->createService(DOGECHAT_SERVICE_UUID); + if (_service == nullptr) { + DOGECHAT_DEBUG_PRINTLN("Failed to create Dogechat service"); + return false; + } + + // Create characteristic with READ, WRITE, WRITE_NR, NOTIFY, and INDICATE properties + _characteristic = _service->createCharacteristic( + DOGECHAT_CHARACTERISTIC_UUID, + BLECharacteristic::PROPERTY_READ | + BLECharacteristic::PROPERTY_WRITE | + BLECharacteristic::PROPERTY_WRITE_NR | + BLECharacteristic::PROPERTY_NOTIFY | + BLECharacteristic::PROPERTY_INDICATE + ); + + if (_characteristic == nullptr) { + DOGECHAT_DEBUG_PRINTLN("Failed to create Dogechat characteristic"); + return false; + } + + // Dogechat uses open security (no PIN required) + _characteristic->setAccessPermissions(ESP_GATT_PERM_READ | ESP_GATT_PERM_WRITE); + + // Add descriptor for notifications + _characteristic->addDescriptor(new BLE2902()); + + // Set callbacks + _characteristic->setCallbacks(this); + + DOGECHAT_DEBUG_PRINTLN("Dogechat BLE service attached to server"); + return true; +} + +// MeshCore UART service UUID (for scan response) +#define MESHCORE_UART_SERVICE_UUID "6E400001-B5A3-F393-E0A9-E50E24DCCA9E" + +void DogechatBLEService::start() { + if (_service == nullptr) { + DOGECHAT_DEBUG_PRINTLN("Cannot start: service not created"); + return; + } + + // Request larger MTU to support Dogechat messages (up to 512 bytes padded) + // This overrides the default MAX_FRAME_SIZE (172) used by MeshCore + BLEDevice::setMTU(517); // Max BLE MTU + + _service->start(); + _serviceActive = true; + + // NOTE: In shared BLE mode (with SerialBLEInterface), we set Dogechat UUID in scan response + // to coexist with MeshCore UUID in main advertisement. + // In standalone mode, caller should use startAdvertising() instead which puts + // Dogechat UUID in main advertisement (required for Dogechat app discovery). + if (_server != nullptr) { + BLEAdvertising* advertising = _server->getAdvertising(); + + // Set scan response data (will be used when advertising starts) + BLEAdvertisementData scanResponse; + scanResponse.setCompleteServices(BLEUUID(DOGECHAT_SERVICE_UUID)); + advertising->setScanResponseData(scanResponse); + + DOGECHAT_DEBUG_PRINTLN("Dogechat BLE service started (shared mode)"); + } else { + DOGECHAT_DEBUG_PRINTLN("Dogechat BLE service started"); + } +} + +void DogechatBLEService::startServiceOnly() { + if (_service == nullptr) { + return; + } + + // Request larger MTU to support Dogechat messages (up to 512 bytes padded) + BLEDevice::setMTU(517); + + _service->start(); + _serviceActive = true; + DOGECHAT_DEBUG_PRINTLN("Dogechat BLE service started (standalone)"); +} + +void DogechatBLEService::setDeviceName(const char* name) { + strncpy(_deviceName, name, sizeof(_deviceName) - 1); + _deviceName[sizeof(_deviceName) - 1] = '\0'; +} + +void DogechatBLEService::startAdvertising() { + if (_server == nullptr) { + return; + } + + BLEAdvertising* advertising = _server->getAdvertising(); + + // Set Dogechat UUID in MAIN advertisement (required for Dogechat app discovery) + // The Dogechat Android app filters on service UUID in main advertisement packet + // BLE advertisement packet is max 31 bytes: + // Flags: 3 bytes, 128-bit UUID: 18 bytes = 21 bytes used + // Remaining for name: 10 bytes (2 header + 8 chars max) + // Put full name in scan response instead + BLEAdvertisementData advData; + advData.setFlags(ESP_BLE_ADV_FLAG_GEN_DISC | ESP_BLE_ADV_FLAG_BREDR_NOT_SPT); + advData.setCompleteServices(BLEUUID(DOGECHAT_SERVICE_UUID)); + // Don't set name in main adv - no room with 128-bit UUID + advertising->setAdvertisementData(advData); + + // Put device name in scan response (ASCII only, no emoji - they break BLE) + char safeName[20]; + size_t j = 0; + for (size_t i = 0; _deviceName[i] != '\0' && j < sizeof(safeName) - 1; i++) { + // Only copy ASCII printable characters (skip emoji/unicode) + if (_deviceName[i] >= 0x20 && _deviceName[i] <= 0x7E) { + safeName[j++] = _deviceName[i]; + } + } + safeName[j] = '\0'; + if (j == 0) strcpy(safeName, "Dogechat"); // Fallback if name was all emoji + BLEAdvertisementData scanResponse; + scanResponse.setName(safeName); + advertising->setScanResponseData(scanResponse); + + advertising->start(); + DOGECHAT_DEBUG_PRINTLN("BLE advertising started: %s", safeName); +} + +void DogechatBLEService::onServerDisconnect() { + checkForDisconnects(); +} + +void DogechatBLEService::clearWriteBuffer() { + _writeBufferOffset = 0; + memset(_writeBuffer, 0, sizeof(_writeBuffer)); +} + +void DogechatBLEService::checkForDisconnects() { + if (_server == nullptr) return; + + uint32_t currentServerCount = _server->getConnectedCount(); + if (currentServerCount < _lastKnownServerCount) { + uint8_t disconnected = _lastKnownServerCount - currentServerCount; + _dogechatClientCount = (disconnected >= _dogechatClientCount) + ? 0 : _dogechatClientCount - disconnected; + _lastKnownServerCount = currentServerCount; + + if (_dogechatClientCount == 0) { + _clientSubscribed = false; + clearWriteBuffer(); + if (_callback != nullptr) { + _callback->onDogechatClientDisconnect(); + } + } + } +} + +bool DogechatBLEService::queueMessage(const DogechatMessage& msg) { + size_t nextTail = (_queueTail + 1) % MESSAGE_QUEUE_SIZE; + + // Check if queue is full + if (nextTail == _queueHead) { + DOGECHAT_DEBUG_PRINTLN("Message queue full, dropping message"); + return false; + } + + _messageQueue[_queueTail].msg = msg; + _messageQueue[_queueTail].valid = true; + _queueTail = nextTail; + + return true; +} + +void DogechatBLEService::processQueue() { + while (_queueHead != _queueTail) { + if (_messageQueue[_queueHead].valid) { + _messageQueue[_queueHead].valid = false; + + if (_callback != nullptr) { + _callback->onDogechatMessageReceived(_messageQueue[_queueHead].msg); + } + } + _queueHead = (_queueHead + 1) % MESSAGE_QUEUE_SIZE; + } +} + +void DogechatBLEService::loop() { + // Check for disconnections (detect when clients drop without callback) + checkForDisconnects(); + + uint32_t now = millis(); + + // Handle deferred connect callback (from BLE callback) + if (_pendingConnect) { + _pendingConnect = false; + if (_callback != nullptr) { + _callback->onDogechatClientConnect(); + } + } + + // Handle deferred data processing (parsing moved out of BLE callback) + // Wait 100ms after last write before processing to allow multi-chunk messages to arrive + if (_pendingData && (now - _lastWriteTime >= 100)) { + _pendingData = false; + DOGECHAT_DEBUG_PRINTLN("Processing %zu buffered bytes", _writeBufferOffset); + + // Try to parse as complete message + DogechatMessage msg; + if (DogechatProtocol::parseMessage(_writeBuffer, _writeBufferOffset, msg)) { + // Successfully parsed - validate and queue + if (DogechatProtocol::validateMessage(msg)) { + DOGECHAT_DEBUG_PRINTLN("Received Dogechat message: type=%02X, len=%d", msg.type, msg.payloadLength); + DOGECHAT_PACKETDUMP("BLE_SERVICE_RX", msg.payload, msg.payloadLength); + queueMessage(msg); + } else { + DOGECHAT_DEBUG_PRINTLN("Invalid Dogechat message received"); + DOGECHAT_PACKETDUMP("BLE_SERVICE_RX_INVALID", _writeBuffer, _writeBufferOffset); + } + clearWriteBuffer(); + } else if (_writeBufferOffset >= DOGECHAT_HEADER_SIZE) { + // Have enough data to check expected size + size_t expectedMin = DogechatProtocol::getMessageSize(msg); + if (_writeBufferOffset > expectedMin + 100) { + DOGECHAT_DEBUG_PRINTLN("Write buffer contains unparseable data, clearing"); + clearWriteBuffer(); + } + // If parse fails but buffer size is reasonable, keep waiting for more data + } + } + + // Check for write buffer timeout + if (_writeBufferOffset > 0) { + if (now - _lastWriteTime > WRITE_TIMEOUT_MS) { + DOGECHAT_DEBUG_PRINTLN("Write buffer timeout, clearing"); + clearWriteBuffer(); + } + } + + // Process queued messages + processQueue(); +} + +void DogechatBLEService::onWrite(BLECharacteristic* pCharacteristic) { + // MINIMAL WORK IN CALLBACK - BLE stack has limited space! + // Just buffer data and set flags; all processing happens in loop() + + std::string value = pCharacteristic->getValue(); + if (value.empty()) { + return; + } + + const uint8_t* data = reinterpret_cast(value.data()); + size_t length = value.length(); + + _lastWriteTime = millis(); + _pendingData = true; // Flag for loop() to process + + // Detect new Dogechat clients by comparing to server's total connection count + if (_server != nullptr) { + uint32_t currentServerCount = _server->getConnectedCount(); + if (currentServerCount > _lastKnownServerCount) { + uint8_t newClients = currentServerCount - _lastKnownServerCount; + _dogechatClientCount += newClients; + _lastKnownServerCount = currentServerCount; + if (newClients > 0) { + _pendingConnect = true; // Defer callback to loop() + } + } + } + + // Append to write buffer (increased size for large messages) + size_t copyLen = length; + if (_writeBufferOffset + copyLen > sizeof(_writeBuffer)) { + clearWriteBuffer(); + copyLen = (length > sizeof(_writeBuffer)) ? sizeof(_writeBuffer) : length; + } + + memcpy(&_writeBuffer[_writeBufferOffset], data, copyLen); + _writeBufferOffset += copyLen; +} + +void DogechatBLEService::onRead(BLECharacteristic* pCharacteristic) { + // Currently unused - reads return the last written value + // NO Serial output here - BLE callback has limited stack +} + +void DogechatBLEService::onStatus(BLECharacteristic* pCharacteristic, Status s, uint32_t code) { + // Called when CCCD is written (client subscribes/unsubscribes to notifications) + // NO Serial output here - BLE callback has limited stack + if (s == Status::SUCCESS_NOTIFY || s == Status::SUCCESS_INDICATE) { + _clientSubscribed = true; + } else if (s == Status::ERROR_NOTIFY_DISABLED) { + _clientSubscribed = false; + } +} + +bool DogechatBLEService::broadcastMessage(const DogechatMessage& msg) { + if (!_serviceActive || _characteristic == nullptr) { + return false; + } + + // Serialize message + uint8_t buffer[DOGECHAT_MAX_MESSAGE_SIZE]; + size_t len = DogechatProtocol::serializeMessage(msg, buffer, sizeof(buffer)); + if (len == 0) { + return false; + } + + DOGECHAT_PACKETDUMP("BLE_SERVICE_TX", buffer, len); + + // Set value and notify + _characteristic->setValue(buffer, len); + _characteristic->notify(true); + + DOGECHAT_DEBUG_PRINTLN("TX: type=0x%02X, len=%zu", msg.type, len); + return true; +} + +#endif // ESP32 diff --git a/src/helpers/dogechat/DogechatBLEService.h b/src/helpers/dogechat/DogechatBLEService.h new file mode 100644 index 000000000..d7fb1ffc5 --- /dev/null +++ b/src/helpers/dogechat/DogechatBLEService.h @@ -0,0 +1,154 @@ +#pragma once + +#include "DogechatProtocol.h" + +#ifdef ESP32 +#include +#include +#include +#include +#include + +/** + * Callback interface for Dogechat BLE events + */ +class DogechatBLECallback { +public: + virtual ~DogechatBLECallback() {} + + /** + * Called when a Dogechat message is received via BLE + * @param msg The received message + */ + virtual void onDogechatMessageReceived(const DogechatMessage& msg) = 0; + + /** + * Called when a Dogechat BLE client connects + */ + virtual void onDogechatClientConnect() {} + + /** + * Called when a Dogechat BLE client disconnects + */ + virtual void onDogechatClientDisconnect() {} +}; + +/** + * Dogechat BLE Service + * Provides a GATT service for Dogechat protocol communication + * Can be attached to an existing BLE server + */ +class DogechatBLEService : public BLECharacteristicCallbacks { +public: + DogechatBLEService(); + + /** + * Attach to an existing BLE server + * Must be called after BLEDevice::init() and server creation + * @param server The BLE server to attach to + * @param callback Callback for Dogechat events + * @return true if service was created successfully + */ + bool attachToServer(BLEServer* server, DogechatBLECallback* callback); + + /** + * Start the Dogechat service (shared BLE mode) + * Sets Dogechat UUID in scan response for coexistence with MeshCore UUID + * Call after attachToServer() and before advertising + */ + void start(); + + /** + * Start the Dogechat service only, without touching advertising + * Use this for standalone mode where startAdvertising() will be called separately + */ + void startServiceOnly(); + + /** + * Set the device name for BLE advertising + * Call before startAdvertising() + */ + void setDeviceName(const char* name); + + /** + * Start BLE advertising with Dogechat UUID in main advertisement + * This is required for Dogechat app discovery (it filters on main adv UUID) + */ + void startAdvertising(); + + /** + * Check if the service is active + */ + bool isActive() const { return _serviceActive; } + + /** + * Check if a Dogechat client is connected + * Note: This tracks clients that have interacted with the Dogechat characteristic + */ + bool hasConnectedClient() const { return _dogechatClientCount > 0; } + + /** + * Broadcast a message to connected Dogechat clients + * @param msg Message to send + * @return true if message was sent + */ + bool broadcastMessage(const DogechatMessage& msg); + + /** + * Process loop - call from main loop + * Handles deferred message processing + */ + void loop(); + + /** + * Mark client as disconnected (call from server disconnect callback) + */ + void onServerDisconnect(); + +protected: + // BLECharacteristicCallbacks + void onWrite(BLECharacteristic* pCharacteristic) override; + void onRead(BLECharacteristic* pCharacteristic) override; + void onStatus(BLECharacteristic* pCharacteristic, Status s, uint32_t code) override; + +private: + BLEServer* _server; + BLEService* _service; + BLECharacteristic* _characteristic; + DogechatBLECallback* _callback; + char _deviceName[48]; + + bool _serviceActive; + uint8_t _dogechatClientCount; // Number of Dogechat clients that have written + uint8_t _lastKnownServerCount; // Track server's total connection count + bool _clientSubscribed; // True when client has subscribed to notifications + + // Flags for deferred processing (to avoid work in BLE callbacks) + volatile bool _pendingConnect; // Client connected, callback not yet called + volatile bool _pendingData; // Data received, not yet parsed + + // Write buffer for reassembling fragmented BLE writes + // Size 512 to handle padded Dogechat messages (Android pads to 256 or 512 bytes) + uint8_t _writeBuffer[512]; + size_t _writeBufferOffset; + uint32_t _lastWriteTime; + static const uint32_t WRITE_TIMEOUT_MS = 5000; + + // Message queue for deferred processing (to avoid processing in BLE callback) + static const size_t MESSAGE_QUEUE_SIZE = 8; + struct QueuedMessage { + DogechatMessage msg; + bool valid; + }; + QueuedMessage _messageQueue[MESSAGE_QUEUE_SIZE]; + volatile size_t _queueHead; + volatile size_t _queueTail; + + void clearWriteBuffer(); + bool queueMessage(const DogechatMessage& msg); + void processQueue(); + void checkForDisconnects(); + +}; + +#endif // ESP32 diff --git a/src/helpers/dogechat/DogechatBridge.cpp b/src/helpers/dogechat/DogechatBridge.cpp new file mode 100644 index 000000000..86dae2090 --- /dev/null +++ b/src/helpers/dogechat/DogechatBridge.cpp @@ -0,0 +1,1649 @@ +#include "DogechatBridge.h" + +#ifdef ARDUINO +#include +#endif + +// Include ed25519 field element operations for Ed25519→Curve25519 conversion +extern "C" { +#include "fe.h" +} + +// Debug output - Adafruit nRF52 core supports Serial.printf +#if DOGECHAT_DEBUG + #define DOGECHAT_DEBUG_PRINTLN(...) do { Serial.printf("DOGECHAT_BRIDGE: "); Serial.printf(__VA_ARGS__); Serial.println(); } while(0) +#else + #define DOGECHAT_DEBUG_PRINTLN(...) {} +#endif + +// Verbose packet hex dump macro (separate from DOGECHAT_DEBUG for optional verbosity) +#if DOGECHAT_DEBUG_PACKETDUMP +static void dumpPacketHex(const char* label, const uint8_t* data, size_t len) { + Serial.printf("PACKETDUMP [%s] (%u bytes):\n", label, (unsigned)len); + for (size_t i = 0; i < len; i++) { + Serial.printf("%02X ", data[i]); + if ((i + 1) % 16 == 0) Serial.println(); + } + if (len % 16 != 0) Serial.println(); +} + #define DOGECHAT_PACKETDUMP(label, data, len) dumpPacketHex(label, data, len) +#else + #define DOGECHAT_PACKETDUMP(label, data, len) {} +#endif + +// PKCS#7 padding for Dogechat protocol signing (must match Android/iOS) +// Block sizes: 256, 512, 1024, 2048 bytes +static size_t applyPKCS7Padding(uint8_t* buffer, size_t dataLen, size_t bufferCapacity) { + // Find optimal block size + static const size_t blockSizes[] = {256, 512, 1024, 2048}; + size_t targetSize = dataLen; // Default to no padding if too large + + for (size_t i = 0; i < 4; i++) { + if (dataLen + 16 <= blockSizes[i]) { // +16 for encryption overhead consideration + targetSize = blockSizes[i]; + break; + } + } + + // Don't pad if already at or exceeding target, or if buffer too small + if (dataLen >= targetSize || targetSize > bufferCapacity) { + return dataLen; + } + + size_t paddingNeeded = targetSize - dataLen; + if (paddingNeeded > 255) { + return dataLen; // PKCS#7 can only encode padding length in 1 byte + } + + // PKCS#7: all padding bytes equal the padding length + for (size_t i = dataLen; i < targetSize; i++) { + buffer[i] = static_cast(paddingNeeded); + } + + return targetSize; +} + +DogechatBridge::DogechatBridge(mesh::Mesh& mesh, mesh::LocalIdentity& identity, const char* nodeName) + : _mesh(mesh) + , _identity(identity) + , _nodeName(nodeName) + , _dogechatPeerId(0) + , _channelConfigured(false) + , _lastAnnounceTime(0) + , _pendingAnnounce(false) + , _timeOffset(0) + , _timeSynced(false) + , _messagesRelayed(0) + , _duplicatesDropped(0) + , _meshChannelConfigured(false) + , _meshChannelIndex(-1) + , _messageHistoryHead(0) +{ + memset(_defaultChannelName, 0, sizeof(_defaultChannelName)); + strcpy(_defaultChannelName, "mesh"); // Default channel + memset(&_meshcoreChannel, 0, sizeof(_meshcoreChannel)); + memset(&_meshChannel, 0, sizeof(_meshChannel)); + memset(_noisePublicKey, 0, sizeof(_noisePublicKey)); + + // Initialize channel mappings + for (size_t i = 0; i < MAX_CHANNEL_MAPPINGS; i++) { + _channelMappings[i].configured = false; + memset(_channelMappings[i].dogechatName, 0, sizeof(_channelMappings[i].dogechatName)); + } + + // Initialize peer cache + for (size_t i = 0; i < PEER_CACHE_SIZE; i++) { + _peerCache[i].valid = false; + _peerCache[i].peerId = 0; + _peerCache[i].timestamp = 0; + memset(_peerCache[i].nickname, 0, sizeof(_peerCache[i].nickname)); + } + + // Initialize fragment buffers + for (size_t i = 0; i < MAX_FRAGMENT_BUFFERS; i++) { + _fragmentBuffers[i].active = false; + _fragmentBuffers[i].senderId = 0; + _fragmentBuffers[i].fragmentId = 0; + _fragmentBuffers[i].totalFragments = 0; + _fragmentBuffers[i].receivedMask = 0; + _fragmentBuffers[i].dataLen = 0; + _fragmentBuffers[i].startTime = 0; + } + + // Initialize message history cache + for (size_t i = 0; i < MESSAGE_HISTORY_SIZE; i++) { + _messageHistory[i].valid = false; + _messageHistory[i].addedTimeMs = 0; + } + + // Initialize pending parts queue + for (size_t i = 0; i < MAX_PENDING_PARTS; i++) { + _pendingParts[i].valid = false; + } + _pendingPartsHead = 0; + _pendingPartsTail = 0; + _lastPartSentTime = 0; +} + +void DogechatBridge::begin() { + // Derive Dogechat peer ID from Meshcore identity + _dogechatPeerId = derivePeerId(_identity); + + // Derive Noise public key (Curve25519) from Ed25519 identity + deriveNoisePublicKey(_identity.pub_key, _noisePublicKey); + + // Configure the #mesh channel for relaying + configureMeshChannel(); + + DOGECHAT_DEBUG_PRINTLN("Bridge initialized, peer ID: %08lX", (unsigned long)(_dogechatPeerId & 0xFFFFFFFF)); +} + +void DogechatBridge::configureMeshChannel() { + // Build the #mesh GroupChannel from the pre-calculated key + memset(&_meshChannel, 0, sizeof(_meshChannel)); + memcpy(_meshChannel.secret, MESH_CHANNEL_KEY, 16); + + // Compute the 1-byte hash used for lookup + mesh::Utils::sha256(_meshChannel.hash, sizeof(_meshChannel.hash), + MESH_CHANNEL_KEY, 16); + + _meshChannelConfigured = true; + _meshChannelIndex = -1; // Not stored in mesh's channel array yet + + // Register the mapping for Dogechat <-> MeshCore + registerChannelMapping("mesh", _meshChannel); + + DOGECHAT_DEBUG_PRINTLN("#mesh channel configured for bridging"); +} + +bool DogechatBridge::isMeshChannel(const mesh::GroupChannel& channel) const { + // Compare the channel secret (key) - first 16 bytes + return memcmp(channel.secret, MESH_CHANNEL_KEY, 16) == 0; +} + +void DogechatBridge::addToMessageHistory(const DogechatMessage& msg) { + _messageHistory[_messageHistoryHead].msg = msg; + _messageHistory[_messageHistoryHead].addedTimeMs = millis(); + _messageHistory[_messageHistoryHead].valid = true; + _messageHistoryHead = (_messageHistoryHead + 1) % MESSAGE_HISTORY_SIZE; +} + +// ============================================================================ +// GCS Filter Implementation for REQUEST_SYNC +// ============================================================================ + +// GCS TLV types (from Android RequestSyncPacket.kt) +#define GCS_TLV_P 0x01 // Golomb-Rice parameter (1 byte) +#define GCS_TLV_N 0x02 // Number of elements (4 bytes BE) +#define GCS_TLV_DATA 0x03 // Encoded bitstream + +bool DogechatBridge::parseGCSFilter(const uint8_t* payload, size_t len, GCSFilter& outFilter) { + // Initialize filter with defaults + outFilter.p = 0; + outFilter.n = 0; + outFilter.m = 0; + outFilter.data = nullptr; + outFilter.dataLen = 0; + + if (payload == nullptr || len < 3) { + return false; + } + + // Parse TLV structure + size_t offset = 0; + bool hasP = false, hasN = false, hasData = false; + + while (offset + 2 <= len) { + uint8_t type = payload[offset++]; + uint8_t length = payload[offset++]; + + if (offset + length > len) { + break; // Truncated TLV + } + + switch (type) { + case GCS_TLV_P: + if (length >= 1) { + outFilter.p = payload[offset]; + hasP = true; + } + break; + + case GCS_TLV_N: + if (length >= 4) { + // Big-endian 4-byte integer + outFilter.n = (static_cast(payload[offset]) << 24) | + (static_cast(payload[offset + 1]) << 16) | + (static_cast(payload[offset + 2]) << 8) | + static_cast(payload[offset + 3]); + hasN = true; + } + break; + + case GCS_TLV_DATA: + outFilter.data = &payload[offset]; + outFilter.dataLen = length; + hasData = true; + break; + + default: + // Unknown TLV type - skip + break; + } + offset += length; + } + + // Calculate M = N * 2^P + if (hasP && hasN) { + outFilter.m = outFilter.n << outFilter.p; + } + + DOGECHAT_DEBUG_PRINTLN("GCS filter: P=%d, N=%u, M=%u, dataLen=%u", + outFilter.p, outFilter.n, outFilter.m, (unsigned)outFilter.dataLen); + + // Filter is valid if we have all required fields + return hasP && hasN && hasData && outFilter.n > 0; +} + +bool DogechatBridge::GCSFilter::mightContain(const uint8_t* packetId16) const { + // Check if a packet ID might be in the GCS filter. + // + // GCS works by hashing items to a value in range [0, M), then Golomb-Rice + // encoding the sorted differences. To check membership, we: + // 1. Hash the packet ID to get value h in [0, M) + // 2. Decode the filter to find all stored values + // 3. Check if h is among the stored values + // + // For efficiency, we decode on-the-fly and stop early if we find or pass h. + + if (data == nullptr || dataLen == 0 || n == 0 || m == 0) { + return false; // Empty filter - nothing matches + } + + // Hash packet ID to range [0, M) using SipHash-like reduction + // Use first 8 bytes of packet ID as uint64, then reduce to M + uint64_t h64 = 0; + for (int i = 0; i < 8; i++) { + h64 |= (static_cast(packetId16[i]) << (i * 8)); + } + + // Reduce to range [0, M) using multiplication and shift (fast modulo) + // h = (h64 * M) >> 64, but since M fits in 32 bits, we use simpler approach + uint32_t h = static_cast(h64 % m); + + // Golomb-Rice decode the filter to check membership + // Each value is encoded as: unary(quotient) + binary(remainder, P bits) + // Values are delta-encoded (differences from previous value) + + uint32_t current = 0; // Running sum of deltas + size_t bitPos = 0; // Bit position in data + + // Helper to read a single bit + auto readBit = [this, &bitPos]() -> int { + if (bitPos / 8 >= dataLen) return -1; // Past end + int bit = (data[bitPos / 8] >> (7 - (bitPos % 8))) & 1; + bitPos++; + return bit; + }; + + // Helper to read P bits as binary value + auto readBinary = [this, &bitPos](uint8_t bits) -> int32_t { + if (bits == 0) return 0; + if (bitPos / 8 + (bits + 7) / 8 > dataLen) return -1; + + int32_t value = 0; + for (uint8_t i = 0; i < bits; i++) { + int bit = (data[bitPos / 8] >> (7 - (bitPos % 8))) & 1; + value = (value << 1) | bit; + bitPos++; + } + return value; + }; + + for (uint32_t i = 0; i < n; i++) { + // Decode quotient (unary: count 1s until 0) + uint32_t quotient = 0; + int bit; + while ((bit = readBit()) == 1) { + quotient++; + if (quotient > m) return false; // Malformed filter + } + if (bit < 0) return false; // Truncated + + // Decode remainder (P bits, binary) + int32_t remainder = readBinary(p); + if (remainder < 0) return false; // Truncated + + // Reconstruct delta and add to current + uint32_t delta = (quotient << p) | static_cast(remainder); + current += delta; + + // Check if we found or passed the target + if (current == h) { + return true; // Found - requester has this message + } + if (current > h) { + return false; // Passed it - requester doesn't have this message + } + } + + return false; // Not found in filter +} + +void DogechatBridge::handleRequestSync(const DogechatMessage& msg) { + DOGECHAT_DEBUG_PRINTLN("REQUEST_SYNC from %08lX", (unsigned long)(msg.getSenderId64() & 0xFFFFFFFF)); + + // Parse the GCS filter from the REQUEST_SYNC payload + // The filter tells us which messages the requester already has + GCSFilter filter; + bool hasFilter = parseGCSFilter(msg.payload, msg.payloadLength, filter); + + if (hasFilter) { + DOGECHAT_DEBUG_PRINTLN("REQUEST_SYNC has GCS filter (N=%u elements)", filter.n); + DOGECHAT_PACKETDUMP("GCS_FILTER", msg.payload, msg.payloadLength); + } else { + DOGECHAT_DEBUG_PRINTLN("REQUEST_SYNC has no GCS filter - sending all"); + } + + // Expire old messages before responding + uint32_t now = millis(); + for (size_t i = 0; i < MESSAGE_HISTORY_SIZE; i++) { + if (_messageHistory[i].valid && + (now - _messageHistory[i].addedTimeMs) > MESSAGE_EXPIRY_MS) { + _messageHistory[i].valid = false; + DOGECHAT_DEBUG_PRINTLN("Expired old message at index %u", (unsigned)i); + } + } + + // Send cached messages that the requester doesn't have + int sent = 0; + int skipped = 0; + for (size_t i = 0; i < MESSAGE_HISTORY_SIZE; i++) { + if (!_messageHistory[i].valid) continue; + + // If we have a filter, check if requester already has this message + if (hasFilter) { + uint8_t packetId[16]; + DogechatProtocol::computePacketId(_messageHistory[i].msg, packetId); + DOGECHAT_PACKETDUMP("SYNC_CHECK_PACKET_ID", packetId, 16); + + if (filter.mightContain(packetId)) { + // Requester likely already has this message - skip it + skipped++; + DOGECHAT_DEBUG_PRINTLN("Skipping msg %u - already in filter", (unsigned)i); + continue; + } + } + +#if defined(ESP32) || defined(NRF52_PLATFORM) + _bleService.broadcastMessage(_messageHistory[i].msg); + sent++; +#endif + } + + // Always send our announcement + sendPeerAnnouncement(); + + DOGECHAT_DEBUG_PRINTLN("REQUEST_SYNC response: sent %d messages, skipped %d (filter=%s)", + sent, skipped, hasFilter ? "yes" : "no"); +} + +uint64_t DogechatBridge::derivePeerId(const mesh::LocalIdentity& identity) { + // Use first 8 bytes of public key as peer ID + uint64_t id = 0; + for (int i = 0; i < 8; i++) { + id |= (static_cast(identity.pub_key[i]) << (i * 8)); + } + return id; +} + +void DogechatBridge::deriveNoisePublicKey(const uint8_t* ed25519PubKey, uint8_t* curve25519PubKey) { + // Convert Ed25519 public key (edwards Y) to Curve25519 public key (montgomery X) + // Formula: montgomeryX = (edwardsY + 1) * inverse(1 - edwardsY) mod p + // This is the standard Ed25519→Curve25519 conversion from RFC 7748 + fe x1, tmp0, tmp1; + + fe_frombytes(x1, ed25519PubKey); + fe_1(tmp1); + fe_add(tmp0, x1, tmp1); // tmp0 = edwardsY + 1 + fe_sub(tmp1, tmp1, x1); // tmp1 = 1 - edwardsY + fe_invert(tmp1, tmp1); // tmp1 = inverse(1 - edwardsY) + fe_mul(x1, tmp0, tmp1); // x1 = (edwardsY + 1) * inverse(1 - edwardsY) + fe_tobytes(curve25519PubKey, x1); +} + +void DogechatBridge::loop() { +#if defined(ESP32) || defined(NRF52_PLATFORM) + _bleService.loop(); + + // Process pending multi-part messages + processPendingParts(); + + uint32_t now = millis(); + + // Handle deferred announcement (from BLE callback - limited stack) + if (_pendingAnnounce) { + _pendingAnnounce = false; + sendPeerAnnouncement(); + _lastAnnounceTime = now; + } + + // Periodically expire old messages from cache (every 30 seconds) + static uint32_t lastExpiryCheck = 0; + if (now - lastExpiryCheck >= 30000) { + lastExpiryCheck = now; + for (size_t i = 0; i < MESSAGE_HISTORY_SIZE; i++) { + if (_messageHistory[i].valid && + (now - _messageHistory[i].addedTimeMs) > MESSAGE_EXPIRY_MS) { + _messageHistory[i].valid = false; + } + } + } + + // Always send periodic announcements - don't check if service is active + // This ensures announcements resume after MeshCore app disconnects + // BLE notification will go out whether or not anyone is listening + // Use shorter interval when we know a client has interacted + uint32_t interval = _bleService.hasConnectedClient() + ? ANNOUNCE_INTERVAL_CONNECTED_MS + : ANNOUNCE_INTERVAL_MS; + + if (now - _lastAnnounceTime >= interval) { + DOGECHAT_DEBUG_PRINTLN("Sending periodic announcement (interval=%lu, elapsed=%lu)", + interval, now - _lastAnnounceTime); + sendPeerAnnouncement(); + _lastAnnounceTime = now; + } +#endif +} + +#if defined(ESP32) +bool DogechatBridge::attachBLEService(BLEServer* server) { + if (!_bleService.attachToServer(server, this)) { + DOGECHAT_DEBUG_PRINTLN("Failed to attach BLE service"); + return false; + } + _bleService.start(); + + // Send first announcement immediately so connecting clients see us right away + sendPeerAnnouncement(); + _lastAnnounceTime = millis(); + + return true; +} + +bool DogechatBridge::beginStandalone(const char* deviceName) { + // Initialize BLE independently (no SerialBLEInterface) + BLEDevice::init(deviceName); + BLEDevice::setMTU(185); + + // Create BLE server + BLEServer* server = BLEDevice::createServer(); + if (server == nullptr) { + return false; + } + + // Attach Dogechat service to the server + if (!_bleService.attachToServer(server, this)) { + return false; + } + + // Set device name and start service (without touching advertising) + _bleService.setDeviceName(deviceName); + _bleService.startServiceOnly(); + + // Start advertising with Dogechat UUID in main advertisement + _bleService.startAdvertising(); + + // Send first announcement + sendPeerAnnouncement(); + _lastAnnounceTime = millis(); + + DOGECHAT_DEBUG_PRINTLN("Standalone mode initialized: %s", deviceName); + return true; +} + +bool DogechatBridge::isBLEActive() const { + return _bleService.isActive(); +} + +bool DogechatBridge::hasDogechatClient() const { + return _bleService.hasConnectedClient(); +} +#elif defined(NRF52_PLATFORM) +bool DogechatBridge::beginStandalone(const char* deviceName) { + // Initialize Bluefruit BLE with Dogechat service + if (!_bleService.beginStandalone(deviceName, this)) { + DOGECHAT_DEBUG_PRINTLN("Failed to start BLE service"); + return false; + } + + // Start advertising with Dogechat UUID in main advertisement + _bleService.startAdvertising(); + + // Send first announcement + sendPeerAnnouncement(); + _lastAnnounceTime = millis(); + + DOGECHAT_DEBUG_PRINTLN("Standalone mode initialized: %s", deviceName); + return true; +} + +bool DogechatBridge::isBLEActive() const { + return _bleService.isActive(); +} + +bool DogechatBridge::hasDogechatClient() const { + return _bleService.hasConnectedClient(); +} +#else +bool DogechatBridge::isBLEActive() const { + return false; +} + +bool DogechatBridge::hasDogechatClient() const { + return false; +} +#endif + +void DogechatBridge::setDefaultChannel(const char* channelName) { + strncpy(_defaultChannelName, channelName, sizeof(_defaultChannelName) - 1); + _defaultChannelName[sizeof(_defaultChannelName) - 1] = '\0'; +} + +void DogechatBridge::setMeshcoreChannel(const mesh::GroupChannel& channel) { + _meshcoreChannel = channel; + _channelConfigured = true; +} + +bool DogechatBridge::registerChannelMapping(const char* dogechatChannelName, const mesh::GroupChannel& meshChannel) { + // Skip # prefix if present + const char* name = dogechatChannelName; + if (name[0] == '#') name++; + + // Check if mapping already exists (update it) + for (size_t i = 0; i < MAX_CHANNEL_MAPPINGS; i++) { + if (_channelMappings[i].configured && + strcmp(_channelMappings[i].dogechatName, name) == 0) { + _channelMappings[i].meshChannel = meshChannel; + return true; + } + } + + // Find empty slot + for (size_t i = 0; i < MAX_CHANNEL_MAPPINGS; i++) { + if (!_channelMappings[i].configured) { + strncpy(_channelMappings[i].dogechatName, name, sizeof(_channelMappings[i].dogechatName) - 1); + _channelMappings[i].dogechatName[sizeof(_channelMappings[i].dogechatName) - 1] = '\0'; + _channelMappings[i].meshChannel = meshChannel; + _channelMappings[i].configured = true; + DOGECHAT_DEBUG_PRINTLN("Registered channel mapping: %s", name); + return true; + } + } + + DOGECHAT_DEBUG_PRINTLN("Channel mapping registry full"); + return false; +} + +bool DogechatBridge::findMeshChannel(const char* channelName, mesh::GroupChannel& outChannel) { + // Skip # prefix if present + const char* name = channelName; + if (name[0] == '#') name++; + + // Search in registry + for (size_t i = 0; i < MAX_CHANNEL_MAPPINGS; i++) { + if (_channelMappings[i].configured && + strcmp(_channelMappings[i].dogechatName, name) == 0) { + outChannel = _channelMappings[i].meshChannel; + return true; + } + } + + // Fall back to default channel if configured + if (_channelConfigured) { + outChannel = _meshcoreChannel; + return true; + } + + return false; +} + +const char* DogechatBridge::getChannelName(const mesh::GroupChannel& channel) { + // Search in registry + for (size_t i = 0; i < MAX_CHANNEL_MAPPINGS; i++) { + if (_channelMappings[i].configured && + memcmp(&_channelMappings[i].meshChannel, &channel, sizeof(mesh::GroupChannel)) == 0) { + return _channelMappings[i].dogechatName; + } + } + + // Return default channel name + return _defaultChannelName; +} + +void DogechatBridge::syncTimeFromPacket(uint64_t packetTimestamp) { +#ifdef ARDUINO + // Only sync if the timestamp looks reasonable (after year 2024, before year 2100) + // Unix timestamp for 2024-01-01 00:00:00 UTC = 1704067200000 ms + // Unix timestamp for 2100-01-01 00:00:00 UTC = 4102444800000 ms + const uint64_t MIN_VALID_TIMESTAMP = 1704067200000ULL; // 2024-01-01 + const uint64_t MAX_VALID_TIMESTAMP = 4102444800000ULL; // 2100-01-01 + + // Threshold for RTC sync - if RTC differs by more than this, update it + // This allows Dogechat to act as an NTP-like time source for the mesh + const uint32_t RTC_SYNC_THRESHOLD_SECS = 30; // 30 seconds + + if (packetTimestamp >= MIN_VALID_TIMESTAMP && packetTimestamp <= MAX_VALID_TIMESTAMP) { + uint64_t localMs = millis(); + int64_t newOffset = static_cast(packetTimestamp) - static_cast(localMs); + + // If this is first sync, or if the offset changed significantly (device was rebooted), update it + if (!_timeSynced || abs(newOffset - _timeOffset) > 60000) { // > 1 minute drift + _timeOffset = newOffset; + _timeSynced = true; + DOGECHAT_DEBUG_PRINTLN("Time synced from Dogechat: offset=%ld ms", (long)(_timeOffset / 1000)); + } + + // Also sync the RTC if the difference is significant + // This allows other MeshCore components to benefit from Dogechat time sync + mesh::RTCClock* rtc = _mesh.getRTCClock(); + if (rtc != nullptr) { + uint32_t dogechatTimeSecs = static_cast(packetTimestamp / 1000ULL); + uint32_t rtcTime = rtc->getCurrentTime(); + int32_t timeDiff = static_cast(dogechatTimeSecs) - static_cast(rtcTime); + + if (abs(timeDiff) > RTC_SYNC_THRESHOLD_SECS) { + rtc->setCurrentTime(dogechatTimeSecs); + DOGECHAT_DEBUG_PRINTLN("RTC synced from Dogechat: %u (was off by %d secs)", + dogechatTimeSecs, timeDiff); + } + } + } +#endif +} + +bool DogechatBridge::parseAnnounceTLV(const uint8_t* payload, size_t len, char* nickname, size_t nickLen) { + // ANNOUNCE payload is TLV encoded: [type:1][length:1][value:N]... + size_t offset = 0; + while (offset + 2 <= len) { + uint8_t type = payload[offset++]; + uint8_t length = payload[offset++]; + if (offset + length > len) break; + + if (type == DOGECHAT_TLV_NICKNAME && length > 0) { + size_t toCopy = (length < nickLen - 1) ? length : nickLen - 1; + memcpy(nickname, &payload[offset], toCopy); + nickname[toCopy] = '\0'; + return true; + } + offset += length; + } + return false; +} + +void DogechatBridge::cachePeer(uint64_t peerId, const char* nickname) { + uint32_t now = millis(); + + // First, check if peer already exists and update it + for (size_t i = 0; i < PEER_CACHE_SIZE; i++) { + if (_peerCache[i].valid && _peerCache[i].peerId == peerId) { + strncpy(_peerCache[i].nickname, nickname, sizeof(_peerCache[i].nickname) - 1); + _peerCache[i].nickname[sizeof(_peerCache[i].nickname) - 1] = '\0'; + _peerCache[i].timestamp = now; + DOGECHAT_DEBUG_PRINTLN("Updated peer cache: %s -> %08lX", nickname, (unsigned long)(peerId & 0xFFFFFFFF)); + return; + } + } + + // Find an empty slot or the oldest entry + size_t targetIdx = 0; + uint32_t oldestTime = UINT32_MAX; + for (size_t i = 0; i < PEER_CACHE_SIZE; i++) { + if (!_peerCache[i].valid) { + targetIdx = i; + break; + } + if (_peerCache[i].timestamp < oldestTime) { + oldestTime = _peerCache[i].timestamp; + targetIdx = i; + } + } + + // Store the new peer + _peerCache[targetIdx].peerId = peerId; + strncpy(_peerCache[targetIdx].nickname, nickname, sizeof(_peerCache[targetIdx].nickname) - 1); + _peerCache[targetIdx].nickname[sizeof(_peerCache[targetIdx].nickname) - 1] = '\0'; + _peerCache[targetIdx].timestamp = now; + _peerCache[targetIdx].valid = true; + DOGECHAT_DEBUG_PRINTLN("Cached new peer: %s -> %08lX", nickname, (unsigned long)(peerId & 0xFFFFFFFF)); +} + +const char* DogechatBridge::lookupPeerNickname(uint64_t peerId) { + for (size_t i = 0; i < PEER_CACHE_SIZE; i++) { + if (_peerCache[i].valid && _peerCache[i].peerId == peerId) { + return _peerCache[i].nickname; + } + } + return nullptr; +} + +// Compile-time timestamp calculation (approximate) +// __DATE__ format: "Jan 16 2025" +// __TIME__ format: "10:30:45" +static uint64_t getCompileTimeMs() { + // Parse __DATE__ and __TIME__ to get approximate compile timestamp + // This is a rough estimate but good enough for our purposes + const char* date = __DATE__; // "Mmm DD YYYY" + const char* time = __TIME__; // "HH:MM:SS" + + // Month lookup + int month = 0; + if (date[0] == 'J' && date[1] == 'a') month = 1; // Jan + else if (date[0] == 'F') month = 2; // Feb + else if (date[0] == 'M' && date[2] == 'r') month = 3; // Mar + else if (date[0] == 'A' && date[1] == 'p') month = 4; // Apr + else if (date[0] == 'M' && date[2] == 'y') month = 5; // May + else if (date[0] == 'J' && date[2] == 'n') month = 6; // Jun + else if (date[0] == 'J' && date[2] == 'l') month = 7; // Jul + else if (date[0] == 'A' && date[1] == 'u') month = 8; // Aug + else if (date[0] == 'S') month = 9; // Sep + else if (date[0] == 'O') month = 10; // Oct + else if (date[0] == 'N') month = 11; // Nov + else if (date[0] == 'D') month = 12; // Dec + + int day = (date[4] == ' ' ? 0 : (date[4] - '0') * 10) + (date[5] - '0'); + int year = (date[7] - '0') * 1000 + (date[8] - '0') * 100 + + (date[9] - '0') * 10 + (date[10] - '0'); + + int hour = (time[0] - '0') * 10 + (time[1] - '0'); + int minute = (time[3] - '0') * 10 + (time[4] - '0'); + int second = (time[6] - '0') * 10 + (time[7] - '0'); + + // Calculate Unix timestamp (simplified, ignoring leap years for rough estimate) + // Days since Unix epoch (Jan 1, 1970) + int daysPerMonth[] = {0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}; + int64_t days = (int64_t)(year - 1970) * 365 + (year - 1969) / 4; // Leap year approximation + for (int m = 1; m < month; m++) { + days += daysPerMonth[m]; + } + days += day - 1; + + uint64_t seconds = (uint64_t)days * 86400ULL + hour * 3600 + minute * 60 + second; + return seconds * 1000ULL; +} + +uint64_t DogechatBridge::getCurrentTimeMs() { +#ifdef ARDUINO + // If we have synced time from a Dogechat client, use that (most reliable) + if (_timeSynced) { + return static_cast(static_cast(millis()) + _timeOffset); + } + + // Fallback: use a hardcoded reasonable timestamp (Jan 1, 2026) + millis + // This is just for bootstrapping - we'll sync properly once we hear from Dogechat peers + // Jan 1, 2026 00:00:00 UTC = 1767225600 seconds + const uint64_t BOOTSTRAP_TIME_MS = 1767225600000ULL; + return BOOTSTRAP_TIME_MS + millis(); +#else + return 0; +#endif +} + +void DogechatBridge::sendPeerAnnouncement() { +#if defined(ESP32) || defined(NRF52_PLATFORM) + uint64_t timestamp = getCurrentTimeMs(); + +#if DOGECHAT_DEBUG + // Print timestamp in seconds (fits in 32-bit for valid times through 2106) + uint32_t timestampSec = (uint32_t)(timestamp / 1000ULL); + Serial.print("DOGECHAT_BRIDGE: Announce: timestamp_sec="); + Serial.print(timestampSec); + Serial.print(" (expected ~1767000000 for Jan 2026), synced="); + Serial.println(_timeSynced ? 1 : 0); +#endif + + // Use static message to avoid stack overflow on NRF52 + static DogechatMessage msg; + DogechatProtocol::createAnnounce( + msg, + _dogechatPeerId, + _nodeName, + _noisePublicKey, // Curve25519 for Noise protocol + _identity.pub_key, // Ed25519 for signatures + timestamp, + DEFAULT_TTL + ); + + // Sign the announce - Android requires signatures + signMessage(msg); + + if (_bleService.broadcastMessage(msg)) { + DOGECHAT_DEBUG_PRINTLN("Sent peer announcement"); + } else { + DOGECHAT_DEBUG_PRINTLN("FAILED to send peer announcement"); + } +#endif +} + +void DogechatBridge::signMessage(DogechatMessage& msg) { +#if defined(ESP32) || defined(NRF52_PLATFORM) + // IMPORTANT: Dogechat protocol signs with TTL=0 and signature flag cleared + // AND applies PKCS#7 padding to match Android/iOS toBinaryDataForSigning() behavior + uint8_t originalTtl = msg.ttl; + msg.ttl = 0; // Fixed TTL for signing (matches SYNC_TTL_HOPS) + msg.setHasSignature(false); // Clear signature flag for signing + + // Use static buffer to avoid stack overflow on NRF52 + static uint8_t signData[512]; + size_t signLen = DogechatProtocol::serializeMessage(msg, signData, sizeof(signData)); + if (signLen > 0) { + // Apply PKCS#7 padding to match Android/iOS block sizes + size_t paddedLen = applyPKCS7Padding(signData, signLen, sizeof(signData)); + + DOGECHAT_DEBUG_PRINTLN("Signing message: %u bytes (padded from %u)", (unsigned)paddedLen, (unsigned)signLen); + + _identity.sign(msg.signature, signData, paddedLen); + } + + // Restore actual TTL and set signature flag for transmission + msg.ttl = originalTtl; + msg.setHasSignature(true); +#endif +} + +// ============================================================================ +// Dogechat → Meshcore +// ============================================================================ + +#if defined(ESP32) || defined(NRF52_PLATFORM) +void DogechatBridge::onDogechatMessageReceived(const DogechatMessage& msg) { + processDogechatMessage(msg); +} + +void DogechatBridge::onDogechatClientConnect() { + DOGECHAT_DEBUG_PRINTLN("Client connected"); + + // Send announcement immediately when client connects + // This is now called from loop() so it's safe to do heavy work + sendPeerAnnouncement(); + _lastAnnounceTime = millis(); +} + +void DogechatBridge::onDogechatClientDisconnect() { + DOGECHAT_DEBUG_PRINTLN("Dogechat client disconnected"); +} +#endif + +void DogechatBridge::processDogechatMessage(const DogechatMessage& msg) { + DOGECHAT_PACKETDUMP("BLE_RX", msg.payload, msg.payloadLength); + + // Sync time from incoming Dogechat packets (Android sends valid Unix timestamps) + // This is critical: our announces will be rejected as stale without valid time + if (msg.timestamp > 0) { + syncTimeFromPacket(msg.timestamp); + } + + // Check for duplicates - log the hash inputs for debugging +#if DOGECHAT_DEBUG_PACKETDUMP + { + uint64_t senderId = msg.getSenderId64(); + uint32_t timestampSecs = static_cast(msg.timestamp / 1000ULL); + Serial.printf("DEDUP_CHECK: sender=%08lX ts_sec=%lu type=%02X payloadLen=%u\n", + (unsigned long)(senderId & 0xFFFFFFFF), (unsigned long)timestampSecs, msg.type, msg.payloadLength); + if (msg.payloadLength > 0) { + size_t hashBytes = msg.payloadLength < 16 ? msg.payloadLength : 16; + DOGECHAT_PACKETDUMP("DEDUP_PAYLOAD_PREFIX", msg.payload, hashBytes); + } + } +#endif + + // Check for duplicates + if (_duplicateCache.isDuplicate(msg)) { + _duplicatesDropped++; + DOGECHAT_DEBUG_PRINTLN("Duplicate message dropped"); + return; + } + + // Handle based on message type + switch (msg.type) { + case DOGECHAT_MSG_MESSAGE: { + char senderNick[64]; + char content[512]; // Increased to handle decompressed messages (up to ~500 bytes) + char channelName[32]; + bool parsedAsTlv = false; + + // First try TLV parsing (some messages might use it) + bool parsed = parseDogechatMessageTLV(msg.payload, msg.payloadLength, + senderNick, sizeof(senderNick), + content, sizeof(content), + channelName, sizeof(channelName)); + if (parsed) { + parsedAsTlv = true; + } + + if (!parsed && msg.payloadLength > 0 && msg.payloadLength < sizeof(content)) { + // TLV parsing failed - treat payload as plain text + // This is the simple format Dogechat uses for channel messages + memcpy(content, msg.payload, msg.payloadLength); + content[msg.payloadLength] = '\0'; + + // Try to look up cached nickname from previous ANNOUNCE + uint64_t senderId = msg.getSenderId64(); + const char* cachedNick = lookupPeerNickname(senderId); + if (cachedNick != nullptr) { + strncpy(senderNick, cachedNick, sizeof(senderNick) - 1); + senderNick[sizeof(senderNick) - 1] = '\0'; + } else { + // Fall back to ID-based nickname + snprintf(senderNick, sizeof(senderNick), "%04X", + (unsigned)(senderId & 0xFFFF)); + } + + // Plain text messages are assumed to be #mesh channel messages + // The outer HAS_RECIPIENT flag doesn't indicate DM for plain text + strcpy(channelName, DOGECHAT_MESH_CHANNEL); + + DOGECHAT_DEBUG_PRINTLN("Plain text message from %s: %s", senderNick, content); + parsed = true; + } + + if (parsed) { + // IMPORTANT: Only relay #mesh channel messages, ignore everything else + // Check if channel matches #mesh (with or without # prefix) + const char* chanToCheck = channelName; + if (chanToCheck[0] == '#') chanToCheck++; + + if (strcmp(chanToCheck, "mesh") != 0) { + DOGECHAT_DEBUG_PRINTLN("Ignoring message to channel '%s' (only #mesh)", channelName); + break; + } + + // Ignore DMs - only check for TLV-parsed messages + // Plain text messages use outer HAS_RECIPIENT for signing, not for DM indication + if (parsedAsTlv && msg.hasRecipient()) { + DOGECHAT_DEBUG_PRINTLN("Ignoring DM (only #mesh channel is bridged)"); + break; + } + + // Add to message history for REQUEST_SYNC responses + addToMessageHistory(msg); + DOGECHAT_DEBUG_PRINTLN("Added message to history cache"); + + // Relay to MeshCore #mesh channel + DOGECHAT_DEBUG_PRINTLN("Relaying message from %s to #mesh", senderNick); + relayChannelMessageToMesh(msg, channelName, senderNick, content); + } else { + DOGECHAT_DEBUG_PRINTLN("Failed to parse MESSAGE payload (len=%u)", msg.payloadLength); + } + break; + } + + case DOGECHAT_MSG_ANNOUNCE: { + // Parse announce to extract and cache peer's nickname + char nickname[16]; + if (parseAnnounceTLV(msg.payload, msg.payloadLength, nickname, sizeof(nickname))) { + uint64_t peerId = msg.getSenderId64(); + cachePeer(peerId, nickname); + DOGECHAT_DEBUG_PRINTLN("Cached peer: %s (%08lX)", nickname, (unsigned long)(peerId & 0xFFFFFFFF)); + } + break; + } + + case DOGECHAT_MSG_PING: + // Respond with PONG + DOGECHAT_DEBUG_PRINTLN("Received ping, sending pong"); + { + // Static to avoid stack overflow on NRF52 + static DogechatMessage pong; + pong.version = DOGECHAT_VERSION; + pong.type = DOGECHAT_MSG_PONG; + pong.ttl = 1; + pong.timestamp = getCurrentTimeMs(); + pong.flags = DOGECHAT_FLAG_HAS_RECIPIENT; + pong.setSenderId64(_dogechatPeerId); + pong.setRecipientId64(msg.getSenderId64()); + pong.payloadLength = 0; +#if defined(ESP32) || defined(NRF52_PLATFORM) + _bleService.broadcastMessage(pong); +#endif + } + break; + + case DOGECHAT_MSG_FILE_TRANSFER: + // File transfers (images, etc.) are not supported on mesh + DOGECHAT_DEBUG_PRINTLN("Skipping file transfer (not supported)"); + break; + + case DOGECHAT_MSG_FRAGMENT_NEW: + case DOGECHAT_MSG_FRAGMENT: + // Fragment messages are used for long text messages (>245 bytes) + // Reassemble and process when complete + handleFragment(msg); + break; + + case DOGECHAT_MSG_REQUEST_SYNC: + handleRequestSync(msg); + Serial.println("DOGECHAT_BRIDGE: handleRequestSync() returned OK"); + break; + + default: + DOGECHAT_DEBUG_PRINTLN("Unhandled message type: 0x%02X", msg.type); + break; + } + Serial.println("DOGECHAT_BRIDGE: processDogechatMessage() COMPLETE"); +} + +bool DogechatBridge::parseDogechatMessageTLV(const uint8_t* payload, size_t payloadLen, + char* senderNick, size_t senderNickLen, + char* content, size_t contentLen, + char* channelName, size_t channelNameLen) { + // Minimum size: flags(1) + timestamp(8) + idLen(1) + senderLen(1) + contentLen(2) = 13 bytes + if (payloadLen < 13 || senderNickLen == 0 || contentLen == 0 || channelNameLen == 0) { + return false; + } + + senderNick[0] = '\0'; + content[0] = '\0'; + channelName[0] = '\0'; + + size_t offset = 0; + + // Read flags byte + uint8_t flags = payload[offset++]; + bool hasOriginalSender = (flags & 0x04) != 0; + bool hasRecipientNickname = (flags & 0x08) != 0; + bool hasSenderPeerID = (flags & 0x10) != 0; + bool hasMentions = (flags & 0x20) != 0; + bool hasChannel = (flags & 0x40) != 0; + bool isEncrypted = (flags & 0x80) != 0; + + // Skip timestamp (8 bytes big-endian) + if (offset + 8 > payloadLen) { + return false; + } + offset += 8; + + // Read ID length and skip ID + if (offset >= payloadLen) { + return false; + } + uint8_t idLen = payload[offset++]; + if (offset + idLen > payloadLen) { + return false; + } + offset += idLen; + + // Read sender nickname + if (offset >= payloadLen) { + return false; + } + uint8_t senderLen = payload[offset++]; + if (offset + senderLen > payloadLen) { + return false; + } + + size_t toCopy = senderLen; + if (toCopy >= senderNickLen) toCopy = senderNickLen - 1; + memcpy(senderNick, &payload[offset], toCopy); + senderNick[toCopy] = '\0'; + offset += senderLen; + + // Read content length (2 bytes big-endian) + if (offset + 2 > payloadLen) { + return false; + } + uint16_t contentLength = (static_cast(payload[offset]) << 8) | payload[offset + 1]; + offset += 2; + + // Read content + if (offset + contentLength > payloadLen) { + return false; + } + if (!isEncrypted) { + toCopy = contentLength; + if (toCopy >= contentLen) toCopy = contentLen - 1; + memcpy(content, &payload[offset], toCopy); + content[toCopy] = '\0'; + } + offset += contentLength; + + // Skip optional fields to get to channel + // Order: originalSender, recipientNickname, senderPeerID, mentions, channel + + if (hasOriginalSender && offset < payloadLen) { + uint8_t len = payload[offset++]; + if (offset + len > payloadLen) return false; + offset += len; + } + + if (hasRecipientNickname && offset < payloadLen) { + uint8_t len = payload[offset++]; + if (offset + len > payloadLen) return false; + offset += len; + } + + if (hasSenderPeerID && offset < payloadLen) { + uint8_t len = payload[offset++]; + if (offset + len > payloadLen) return false; + offset += len; + } + + if (hasMentions && offset < payloadLen) { + uint8_t mentionCount = payload[offset++]; + for (uint8_t i = 0; i < mentionCount && offset < payloadLen; i++) { + uint8_t len = payload[offset++]; + if (offset + len > payloadLen) return false; + offset += len; + } + } + + // Read channel if present + if (hasChannel && offset < payloadLen) { + uint8_t chanLen = payload[offset++]; + if (offset + chanLen > payloadLen) return false; + + toCopy = chanLen; + if (toCopy >= channelNameLen) toCopy = channelNameLen - 1; + memcpy(channelName, &payload[offset], toCopy); + channelName[toCopy] = '\0'; + } + + DOGECHAT_DEBUG_PRINTLN("TLV parsed: sender='%s' content='%s' channel='%s'", senderNick, content, channelName); + + return senderNick[0] != '\0'; // At minimum we need a sender +} + +void DogechatBridge::sendSingleMessageToMesh(const char* senderNick, const char* text) { + // This is the internal function that sends a single message chunk to the mesh. + // The caller is responsible for message splitting if needed. + + // Must have #mesh channel configured + if (!_meshChannelConfigured) { + DOGECHAT_DEBUG_PRINTLN("#mesh channel not configured, cannot send to mesh"); + return; + } + + // Use the #mesh channel for all bridged messages + mesh::GroupChannel targetChannel = _meshChannel; + + // Get timestamp - prefer synced Dogechat time over RTC + // MeshCore uses Unix seconds, Dogechat uses Unix milliseconds + uint32_t timestamp = 0; + if (_timeSynced) { + // Use synced Dogechat time (convert from ms to seconds) + timestamp = static_cast(getCurrentTimeMs() / 1000ULL); + } else { + mesh::RTCClock* rtc = _mesh.getRTCClock(); + timestamp = rtc ? rtc->getCurrentTime() : 0; + } + + // Build Meshcore group message payload + // Format: timestamp(4) + txt_type(1) + "📱 sender: text" + uint8_t payload[MAX_PACKET_PAYLOAD]; + size_t offset = 0; + + // Timestamp (4 bytes) + memcpy(&payload[offset], ×tamp, 4); + offset += 4; + + // Text type (0 = plain text) + payload[offset++] = 0; + + // Add 📱 prefix to sender name (identifies Dogechat origin) + // 📱 = UTF-8: F0 9F 93 B1 (4 bytes) + char prefixedSender[68]; // 4 bytes emoji + 1 space + 63 chars max + snprintf(prefixedSender, sizeof(prefixedSender), "\xF0\x9F\x93\xB1 %s", senderNick); + + size_t senderLen = strlen(prefixedSender); + size_t textLen = strlen(text); + + // Copy "📱 sender: " + size_t available = MAX_PACKET_PAYLOAD - offset - 1; + size_t toCopy = senderLen; + if (toCopy > available) toCopy = available; + memcpy(&payload[offset], prefixedSender, toCopy); + offset += toCopy; + + if (offset < MAX_PACKET_PAYLOAD - 2) { + payload[offset++] = ':'; + payload[offset++] = ' '; + } + + // Copy text + available = MAX_PACKET_PAYLOAD - offset - 1; + toCopy = textLen; + if (toCopy > available) toCopy = available; + memcpy(&payload[offset], text, toCopy); + offset += toCopy; + + // Null terminate + payload[offset] = '\0'; + + DOGECHAT_PACKETDUMP("MESH_TX", payload, offset); + + // Create and send packet immediately (no delay - use pending parts queue for multi-part) + mesh::Packet* pkt = _mesh.createGroupDatagram(PAYLOAD_TYPE_GRP_TXT, targetChannel, payload, offset); + if (pkt != nullptr) { + _mesh.sendFlood(pkt); // Send immediately (no delay) + _messagesRelayed++; + DOGECHAT_DEBUG_PRINTLN("Sent to mesh: %s: %s", prefixedSender, text); + } else { + DOGECHAT_DEBUG_PRINTLN("Failed to create mesh packet (pool may be full)"); + } +} + +bool DogechatBridge::queueMessagePart(const char* senderNick, const char* text) { + // Find next available slot in circular queue + size_t nextTail = (_pendingPartsTail + 1) % MAX_PENDING_PARTS; + if (nextTail == _pendingPartsHead && _pendingParts[_pendingPartsTail].valid) { + // Queue is full + DOGECHAT_DEBUG_PRINTLN("Pending parts queue full, dropping part"); + return false; + } + + strncpy(_pendingParts[_pendingPartsTail].senderNick, senderNick, + sizeof(_pendingParts[_pendingPartsTail].senderNick) - 1); + _pendingParts[_pendingPartsTail].senderNick[sizeof(_pendingParts[_pendingPartsTail].senderNick) - 1] = '\0'; + + strncpy(_pendingParts[_pendingPartsTail].text, text, + sizeof(_pendingParts[_pendingPartsTail].text) - 1); + _pendingParts[_pendingPartsTail].text[sizeof(_pendingParts[_pendingPartsTail].text) - 1] = '\0'; + + _pendingParts[_pendingPartsTail].valid = true; + _pendingPartsTail = nextTail; + + DOGECHAT_DEBUG_PRINTLN("Queued message part for delayed sending"); + return true; +} + +void DogechatBridge::processPendingParts() { + // Check if we have pending parts to send + if (_pendingPartsHead == _pendingPartsTail && !_pendingParts[_pendingPartsHead].valid) { + return; // Queue is empty + } + + // Check if enough time has passed since last part was sent + uint32_t now = millis(); + if (now - _lastPartSentTime < PART_SEND_DELAY_MS) { + return; // Not time yet + } + + // Send the next part + if (_pendingParts[_pendingPartsHead].valid) { + DOGECHAT_DEBUG_PRINTLN("Sending queued part: %s", _pendingParts[_pendingPartsHead].text); + sendSingleMessageToMesh(_pendingParts[_pendingPartsHead].senderNick, + _pendingParts[_pendingPartsHead].text); + _pendingParts[_pendingPartsHead].valid = false; + _pendingPartsHead = (_pendingPartsHead + 1) % MAX_PENDING_PARTS; + _lastPartSentTime = now; + } +} + +void DogechatBridge::relayChannelMessageToMesh(const DogechatMessage& msg, const char* channelName, + const char* senderNick, const char* text) { + // Find the MeshCore channel for this Dogechat channel + mesh::GroupChannel targetChannel; + if (!findMeshChannel(channelName, targetChannel)) { + DOGECHAT_DEBUG_PRINTLN("No channel mapping for '%s' - check registerChannelMapping()", channelName); + return; + } + + // Calculate available space for message text + // MeshCore MAX_TEXT_LEN is 160 bytes total for: "📱nick: text" + // Overhead: 📱(4) + space(1) + nick(up to 13) + ": "(2) = ~20 bytes + // With part indicator "[X/Y] "(7 bytes), we have ~133 bytes for text + // Be conservative and use 120 bytes per chunk + const size_t MAX_CHUNK_SIZE = 120; + + size_t contentLen = strlen(text); + + if (contentLen <= MAX_CHUNK_SIZE) { + // Single message - no splitting needed + sendSingleMessageToMesh(senderNick, text); + return; + } + + // Calculate number of parts needed + int numParts = (contentLen + MAX_CHUNK_SIZE - 1) / MAX_CHUNK_SIZE; + if (numParts > 9) { + numParts = 9; // Cap at 9 parts to keep indicator short + } + + DOGECHAT_DEBUG_PRINTLN("Splitting message from %s into %d parts (len=%d)", senderNick, numParts, (int)contentLen); + + // Send part 1 immediately, queue remaining parts for delayed sending + // This avoids overwhelming the mesh packet pool which can silently drop delayed packets + size_t offset = 0; + for (int part = 0; part < numParts && offset < contentLen; part++) { + size_t remaining = contentLen - offset; + size_t chunkLen = (remaining > MAX_CHUNK_SIZE) ? MAX_CHUNK_SIZE : remaining; + + // Adjust chunk length to avoid splitting UTF-8 multibyte characters + // UTF-8 continuation bytes start with 10xxxxxx (0x80-0xBF) + while (chunkLen > 0 && chunkLen < remaining) { + uint8_t nextByte = (uint8_t)text[offset + chunkLen]; + if ((nextByte & 0xC0) != 0x80) { + // This is not a continuation byte, safe to split here + break; + } + // Back up to avoid splitting mid-character + chunkLen--; + } + + if (chunkLen == 0) { + DOGECHAT_DEBUG_PRINTLN("Error: Could not find safe UTF-8 split point"); + break; + } + + // Build chunk with part indicator + char chunk[180]; // Room for part indicator + text + snprintf(chunk, sizeof(chunk), "[%d/%d] %.*s", + part + 1, numParts, (int)chunkLen, text + offset); + + DOGECHAT_DEBUG_PRINTLN("Part %d/%d: offset=%d, len=%d", + part + 1, numParts, (int)offset, (int)chunkLen); + DOGECHAT_PACKETDUMP("SPLIT_PART", (const uint8_t*)chunk, strlen(chunk)); + + if (part == 0) { + // Send first part immediately + sendSingleMessageToMesh(senderNick, chunk); + _lastPartSentTime = millis(); // Start the timer for subsequent parts + } else { + // Queue remaining parts for delayed sending via processPendingParts() + queueMessagePart(senderNick, chunk); + } + + offset += chunkLen; + } +} + +void DogechatBridge::relayDirectMessageToMesh(const DogechatMessage& msg, const char* text) { + // For DMs, we need to find the recipient in Meshcore contacts + // This requires integration with the contact database + // For now, log and skip + DOGECHAT_DEBUG_PRINTLN("DM relay not yet implemented - need contact lookup"); + + // TODO: Implement DM relay + // 1. Map Dogechat recipient ID to Meshcore Identity + // 2. Look up shared secret + // 3. Create encrypted TXT_MSG packet + // 4. Send via appropriate routing +} + +// ============================================================================ +// Fragment Reassembly +// ============================================================================ + +void DogechatBridge::handleFragment(const DogechatMessage& msg) { + // Fragment header format (from Dogechat protocol): + // [fragmentId:1][totalFragments:1][fragmentIndex:1][data...] + if (msg.payloadLength < 3) { + return; + } + + uint8_t fragmentId = msg.payload[0]; + uint8_t totalFragments = msg.payload[1]; + uint8_t fragmentIndex = msg.payload[2]; + + uint64_t senderId = msg.getSenderId64(); + + // Validate fragment parameters + if (totalFragments == 0 || totalFragments > 8 || fragmentIndex >= totalFragments) { + return; + } + + uint32_t now = millis(); + + // Clean up expired fragment buffers + for (size_t i = 0; i < MAX_FRAGMENT_BUFFERS; i++) { + if (_fragmentBuffers[i].active && + (now - _fragmentBuffers[i].startTime) > FRAGMENT_TIMEOUT_MS) { + _fragmentBuffers[i].active = false; + } + } + + // Find existing buffer for this sender/fragmentId + FragmentBuffer* buf = nullptr; + for (size_t i = 0; i < MAX_FRAGMENT_BUFFERS; i++) { + if (_fragmentBuffers[i].active && + _fragmentBuffers[i].senderId == senderId && + _fragmentBuffers[i].fragmentId == fragmentId) { + buf = &_fragmentBuffers[i]; + break; + } + } + + // New fragment sequence - find empty buffer + if (buf == nullptr) { + if (msg.type != DOGECHAT_MSG_FRAGMENT_NEW && fragmentIndex != 0) { + // Missed the first fragment - can't reassemble + DOGECHAT_DEBUG_PRINTLN("Fragment sequence interrupted - missed first fragment (idx=%u)", fragmentIndex); + return; + } + + for (size_t i = 0; i < MAX_FRAGMENT_BUFFERS; i++) { + if (!_fragmentBuffers[i].active) { + buf = &_fragmentBuffers[i]; + buf->active = true; + buf->senderId = senderId; + buf->fragmentId = fragmentId; + buf->totalFragments = totalFragments; + buf->receivedMask = 0; + buf->dataLen = 0; + buf->startTime = now; + memset(buf->data, 0, sizeof(buf->data)); + break; + } + } + } + + if (buf == nullptr) { + DOGECHAT_DEBUG_PRINTLN("Fragment buffer full - cannot reassemble (sender=%08lX)", (unsigned long)(senderId & 0xFFFFFFFF)); + return; + } + + // Copy fragment data + size_t dataOffset = 3; // Skip header + size_t dataLen = msg.payloadLength - dataOffset; + + // Each fragment contains ~240 bytes of data (245 - 3 header - 2 checksum) + size_t fragmentDataSize = 240; + size_t insertOffset = fragmentIndex * fragmentDataSize; + + if (insertOffset + dataLen > sizeof(buf->data)) { + return; + } + + memcpy(&buf->data[insertOffset], &msg.payload[dataOffset], dataLen); + buf->receivedMask |= (1 << fragmentIndex); + + // Track total data length + size_t endPos = insertOffset + dataLen; + if (endPos > buf->dataLen) { + buf->dataLen = endPos; + } + + DOGECHAT_DEBUG_PRINTLN("Fragment %d/%d stored", fragmentIndex + 1, totalFragments); + + // Check if complete + uint8_t expectedMask = (1 << totalFragments) - 1; + if (buf->receivedMask == expectedMask) { + // Reassembly complete! + DOGECHAT_DEBUG_PRINTLN("Fragment reassembly complete (%u bytes)", (unsigned)buf->dataLen); + + // Create synthetic MESSAGE from reassembled data - static to avoid stack overflow + static DogechatMessage reassembled; + reassembled.version = msg.version; + reassembled.type = DOGECHAT_MSG_MESSAGE; + reassembled.ttl = msg.ttl; + reassembled.timestamp = msg.timestamp; + reassembled.flags = msg.flags; + memcpy(reassembled.senderId, msg.senderId, 8); + memcpy(reassembled.recipientId, msg.recipientId, 8); + + // Copy reassembled data to payload + // Note: For very long messages, we may need to split into multiple mesh messages + size_t copyLen = buf->dataLen; + if (copyLen > DOGECHAT_MAX_PAYLOAD_SIZE) { + copyLen = DOGECHAT_MAX_PAYLOAD_SIZE; + } + memcpy(reassembled.payload, buf->data, copyLen); + reassembled.payloadLength = static_cast(copyLen); + + // Release buffer before processing (in case processing takes time) + buf->active = false; + + // Process the reassembled message + processDogechatMessage(reassembled); + } +} + +// ============================================================================ +// Meshcore → Dogechat +// ============================================================================ + +void DogechatBridge::onMeshcoreGroupMessage(const mesh::GroupChannel& channel, uint32_t timestamp, + const char* senderName, const char* text) { +#if defined(ESP32) || defined(NRF52_PLATFORM) + Serial.println("DOGECHAT_BRIDGE: >>> onMeshcoreGroupMessage() ENTRY <<<"); + DOGECHAT_DEBUG_PRINTLN("MESH_RX: sender=%s text=%s", senderName, text ? text : "(null)"); + DOGECHAT_DEBUG_PRINTLN("MESH_RX: hasClient=%d, BLEactive=%d", + _bleService.hasConnectedClient() ? 1 : 0, + _bleService.isActive() ? 1 : 0); + if (text != nullptr) { + DOGECHAT_PACKETDUMP("MESH_RX", (const uint8_t*)text, strlen(text)); + } + + // IMPORTANT: Only relay #mesh channel messages to Dogechat + if (!isMeshChannel(channel)) { + // Not the #mesh channel - don't relay + DOGECHAT_DEBUG_PRINTLN("Filtering non-#mesh MeshCore group message"); + return; + } + + // Check if this message originated from Dogechat (has phone emoji prefix) + // to prevent rebroadcast loops. UTF-8 phone emoji (📱) is 4 bytes: 0xF0 0x9F 0x93 0xB1 + if (text != nullptr && strlen(text) >= 4) { + if ((uint8_t)text[0] == 0xF0 && (uint8_t)text[1] == 0x9F && + (uint8_t)text[2] == 0x93 && (uint8_t)text[3] == 0xB1) { + DOGECHAT_DEBUG_PRINTLN("Skipping relay - message originated from Dogechat"); + return; + } + } + + // Build simple message content: " text" + // Dogechat displays MESSAGE payload as plain text + // Use static buffers to avoid stack overflow on NRF52 + static char fullContent[200]; + snprintf(fullContent, sizeof(fullContent), "<%s> %s", senderName, text); + + // Create Dogechat message - static to avoid stack overflow + static DogechatMessage msg; + msg.version = DOGECHAT_VERSION; + msg.type = DOGECHAT_MSG_MESSAGE; + msg.ttl = DEFAULT_TTL; + msg.timestamp = getCurrentTimeMs(); + msg.flags = 0; // No special flags - simple channel message + msg.setSenderId64(_dogechatPeerId); + + // Simple payload format - just copy the text content directly + size_t contentLen = strlen(fullContent); + if (contentLen > DOGECHAT_MAX_PAYLOAD_SIZE) { + contentLen = DOGECHAT_MAX_PAYLOAD_SIZE; + } + memcpy(msg.payload, fullContent, contentLen); + msg.payloadLength = static_cast(contentLen); + + // Sign the message + signMessage(msg); + + // Add to message history for REQUEST_SYNC responses + addToMessageHistory(msg); + + DOGECHAT_PACKETDUMP("BLE_TX_FROM_MESH", msg.payload, msg.payloadLength); + bool sent = _bleService.broadcastMessage(msg); + DOGECHAT_DEBUG_PRINTLN("TX to Dogechat: %s (result=%d)", senderName, sent ? 1 : 0); + Serial.println("DOGECHAT_BRIDGE: <<< onMeshcoreGroupMessage() EXIT <<<"); +#endif +} + +void DogechatBridge::onMeshcoreDirectMessage(const uint8_t* senderPubKey, uint32_t timestamp, const char* text) { +#if defined(ESP32) || defined(NRF52_PLATFORM) + if (!_bleService.hasConnectedClient()) { + return; + } + + // Derive sender's Dogechat ID from their public key + uint64_t senderId = 0; + for (int i = 0; i < 8; i++) { + senderId |= (static_cast(senderPubKey[i]) << (i * 8)); + } + + // Create Dogechat DM - static to avoid stack overflow on NRF52 + static DogechatMessage msg; + msg.version = DOGECHAT_VERSION; + msg.type = DOGECHAT_MSG_MESSAGE; + msg.ttl = DEFAULT_TTL; + msg.timestamp = static_cast(timestamp) * 1000ULL; + msg.flags = DOGECHAT_FLAG_HAS_RECIPIENT; + msg.setSenderId64(senderId); + msg.setRecipientId64(_dogechatPeerId); // Recipient is us (relaying to BLE client) + + size_t textLen = strlen(text); + if (textLen > DOGECHAT_MAX_PAYLOAD_SIZE) textLen = DOGECHAT_MAX_PAYLOAD_SIZE; + memcpy(msg.payload, text, textLen); + msg.payloadLength = static_cast(textLen); + + _bleService.broadcastMessage(msg); + DOGECHAT_DEBUG_PRINTLN("Sent DM to Dogechat from %08lX", (unsigned long)(senderId & 0xFFFFFFFF)); +#endif +} + +void DogechatBridge::onMeshcoreAdvert(const mesh::Identity& id, uint32_t timestamp, + const uint8_t* appData, size_t appDataLen) { +#if defined(ESP32) || defined(NRF52_PLATFORM) + if (!_bleService.hasConnectedClient()) { + return; + } + + // Convert Meshcore advert to Dogechat announce + uint64_t peerId = 0; + for (int i = 0; i < 8; i++) { + peerId |= (static_cast(id.pub_key[i]) << (i * 8)); + } + + // Extract name from app data if available + const char* name = "Unknown"; + if (appData != nullptr && appDataLen > 0) { + // Meshcore advert app_data often contains the node name + // This depends on how the advert was created + name = reinterpret_cast(appData); + } + + // Derive Curve25519 key from the peer's Ed25519 key + static uint8_t peerNoiseKey[32]; + deriveNoisePublicKey(id.pub_key, peerNoiseKey); + + // Static to avoid stack overflow on NRF52 + static DogechatMessage msg; + DogechatProtocol::createAnnounce( + msg, + peerId, + name, + peerNoiseKey, // Curve25519 for Noise protocol + id.pub_key, // Ed25519 for signatures + static_cast(timestamp) * 1000ULL, + DEFAULT_TTL + ); + + _bleService.broadcastMessage(msg); + DOGECHAT_DEBUG_PRINTLN("Sent Meshcore advert to Dogechat: %08lX", (unsigned long)(peerId & 0xFFFFFFFF)); +#endif +} + +void DogechatBridge::broadcastToDogechat(const DogechatMessage& msg) { +#if defined(ESP32) || defined(NRF52_PLATFORM) + _bleService.broadcastMessage(msg); +#endif +} diff --git a/src/helpers/dogechat/DogechatBridge.h b/src/helpers/dogechat/DogechatBridge.h new file mode 100644 index 000000000..77ab81f7f --- /dev/null +++ b/src/helpers/dogechat/DogechatBridge.h @@ -0,0 +1,480 @@ +#pragma once + +#include "DogechatProtocol.h" + +#if defined(ESP32) +#include "DogechatBLEService.h" +#elif defined(NRF52_PLATFORM) +#include "../nrf52/DogechatBLEService.h" +#endif + +#include +#include + +// #mesh channel key: first 16 bytes of SHA256("#mesh") +// This is a "hashtag room" where the key is derived from the channel name +// Calculation: SHA256("#mesh").substring(0, 32) = "5b664cde0b08b220612113db980650f3" +static const uint8_t MESH_CHANNEL_KEY[16] = { + 0x5b, 0x66, 0x4c, 0xde, 0x0b, 0x08, 0xb2, 0x20, + 0x61, 0x21, 0x13, 0xdb, 0x98, 0x06, 0x50, 0xf3 +}; + +// Dogechat channel name for #mesh (includes the # prefix) +#define DOGECHAT_MESH_CHANNEL "#mesh" + +/** + * Dogechat Bridge - Translation layer between Dogechat and Meshcore protocols + * + * This bridge relays messages between Dogechat #mesh channel and MeshCore #mesh channel. + * Only #mesh channel messages are relayed - DMs and other channels are ignored. + * The #mesh channel uses a hashtag-derived key: SHA256("#mesh")[0:16] + */ +class DogechatBridge +#if defined(ESP32) || defined(NRF52_PLATFORM) + : public DogechatBLECallback +#endif +{ +public: + /** + * Constructor + * @param mesh Reference to the Mesh instance + * @param identity Reference to this node's LocalIdentity + * @param nodeName This node's display name + */ + DogechatBridge(mesh::Mesh& mesh, mesh::LocalIdentity& identity, const char* nodeName); + + /** + * Initialize the bridge + * Call after mesh.begin() + */ + void begin(); + + /** + * Main loop - call from main loop + */ + void loop(); + +#if defined(ESP32) + /** + * Attach BLE service to existing server (shared BLE mode) + * @param server BLE server to attach to + * @return true if successful + */ + bool attachBLEService(BLEServer* server); + + /** + * Initialize BLE independently (standalone mode, no SerialBLEInterface) + * Creates own BLE server with Dogechat service only. + * Use this when MeshCore companion uses USB serial instead of BLE. + * @param deviceName BLE device name for advertising + * @return true if successful + */ + bool beginStandalone(const char* deviceName); + + /** + * Get the BLE service for disconnect callback registration + */ + DogechatBLEService& getBLEService() { return _bleService; } +#elif defined(NRF52_PLATFORM) + /** + * Initialize BLE independently (standalone mode) + * For NRF52, this uses Bluefruit BLE stack. + * @param deviceName BLE device name for advertising + * @return true if successful + */ + bool beginStandalone(const char* deviceName); + + /** + * Get the BLE service for disconnect callback registration + */ + DogechatBLEService& getBLEService() { return _bleService; } +#endif + + /** + * Handle incoming Meshcore GROUP message + * Call this from onGroupDataRecv() or onChannelMessageRecv() + * @param channel The Meshcore channel + * @param timestamp Message timestamp + * @param senderName Sender's display name (from message) + * @param text Message text + */ + void onMeshcoreGroupMessage(const mesh::GroupChannel& channel, uint32_t timestamp, + const char* senderName, const char* text); + + /** + * Handle incoming Meshcore direct message + * Call this from onPeerDataRecv() + * @param senderPubKey Sender's public key + * @param timestamp Message timestamp + * @param text Message text + */ + void onMeshcoreDirectMessage(const uint8_t* senderPubKey, uint32_t timestamp, const char* text); + + /** + * Handle incoming Meshcore advertisement + * Call this from onAdvertRecv() + * @param id Sender's identity + * @param timestamp Advertisement timestamp + * @param appData Additional data from advertisement + * @param appDataLen Length of additional data + */ + void onMeshcoreAdvert(const mesh::Identity& id, uint32_t timestamp, + const uint8_t* appData, size_t appDataLen); + + /** + * Set the default channel name for Dogechat + * @param channelName Channel name without # prefix + */ + void setDefaultChannel(const char* channelName); + + /** + * Set the channel for outgoing Dogechat messages + * @param channel Meshcore GroupChannel to use + */ + void setMeshcoreChannel(const mesh::GroupChannel& channel); + + /** + * Get this node's Dogechat peer ID (derived from identity) + */ + uint64_t getDogechatPeerId() const { return _dogechatPeerId; } + + /** + * Register a channel mapping between Dogechat channel name and MeshCore GroupChannel + * @param dogechatChannelName Channel name without # prefix (e.g., "general") + * @param meshChannel MeshCore GroupChannel + * @return true if mapping was added (false if registry full) + */ + bool registerChannelMapping(const char* dogechatChannelName, const mesh::GroupChannel& meshChannel); + + /** + * Find MeshCore channel for a Dogechat channel name + * @param channelName Channel name (with or without # prefix) + * @param outChannel OUT: MeshCore GroupChannel if found + * @return true if mapping found + */ + bool findMeshChannel(const char* channelName, mesh::GroupChannel& outChannel); + + /** + * Get Dogechat channel name for a MeshCore channel + * @param channel MeshCore GroupChannel + * @return Channel name (without #) or nullptr if not found + */ + const char* getChannelName(const mesh::GroupChannel& channel); + + /** + * Check if BLE service is active + */ + bool isBLEActive() const; + + /** + * Check if a Dogechat client is connected + */ + bool hasDogechatClient() const; + + /** + * Get statistics + */ + uint32_t getMessagesRelayed() const { return _messagesRelayed; } + uint32_t getDuplicatesDropped() const { return _duplicatesDropped; } + +protected: +#if defined(ESP32) || defined(NRF52_PLATFORM) + // DogechatBLECallback implementation + void onDogechatMessageReceived(const DogechatMessage& msg) override; + void onDogechatClientConnect() override; + void onDogechatClientDisconnect() override; +#endif + +private: + mesh::Mesh& _mesh; + mesh::LocalIdentity& _identity; + const char* _nodeName; + +#if defined(ESP32) || defined(NRF52_PLATFORM) + DogechatBLEService _bleService; +#endif + + DogechatDuplicateCache _duplicateCache; + + // Dogechat peer identity (derived from Meshcore identity) + uint64_t _dogechatPeerId; + + // Noise public key (Curve25519, derived from Ed25519 identity) + uint8_t _noisePublicKey[32]; + + // Default channel for Dogechat messages + char _defaultChannelName[32]; + mesh::GroupChannel _meshcoreChannel; + bool _channelConfigured; + + // Channel registry for bidirectional mapping + struct ChannelMapping { + char dogechatName[32]; // Channel name without # prefix + mesh::GroupChannel meshChannel; + bool configured; + }; + static const size_t MAX_CHANNEL_MAPPINGS = 4; + ChannelMapping _channelMappings[MAX_CHANNEL_MAPPINGS]; + + // Announcement timing + uint32_t _lastAnnounceTime; + volatile bool _pendingAnnounce; // Flag to defer announcement to main loop (BLE callback has limited stack) + static const uint32_t ANNOUNCE_INTERVAL_MS = 5000; // 5 seconds when idle + static const uint32_t ANNOUNCE_INTERVAL_CONNECTED_MS = 3000; // 3 seconds when client connected + + // Time synchronization (calibrated from received Dogechat packets) + // Android sends Unix timestamps; we sync from them since ESP32 may not have valid RTC + int64_t _timeOffset; // Offset to add to millis() to get Unix time (ms) + bool _timeSynced; // True after receiving at least one valid timestamp from Android + + // Statistics + uint32_t _messagesRelayed; + uint32_t _duplicatesDropped; + + // TTL for outgoing messages + static const uint8_t DEFAULT_TTL = 8; + + // #mesh channel configuration + mesh::GroupChannel _meshChannel; // The MeshCore #mesh channel + bool _meshChannelConfigured; // True if #mesh channel is found/configured + int _meshChannelIndex; // Index in the mesh's channel array (-1 if not found) + + // Message history cache for REQUEST_SYNC + // Stores recent messages so we can respond to sync requests + struct CachedMessage { + DogechatMessage msg; + uint32_t addedTimeMs; // millis() when message was cached (for expiration) + bool valid; + }; + static const size_t MESSAGE_HISTORY_SIZE = 16; + static const uint32_t MESSAGE_EXPIRY_MS = 300000; // 5 minutes + CachedMessage _messageHistory[MESSAGE_HISTORY_SIZE]; + size_t _messageHistoryHead; + + /** + * Golomb-Coded Set (GCS) filter for REQUEST_SYNC + * Used to determine which messages the requester already has. + * See Android RequestSyncPacket.kt for format details. + */ + struct GCSFilter { + uint8_t p; // Golomb-Rice parameter (bits for remainder) + uint32_t n; // Number of elements in filter + uint32_t m; // Range M = N * 2^P + const uint8_t* data; // Pointer to encoded bitstream + size_t dataLen; // Length of encoded data + + /** + * Check if a packet ID might be in the filter (probabilistic) + * @param packetId16 16-byte packet ID + * @return true if the ID might be in the filter (requester may have it) + */ + bool mightContain(const uint8_t* packetId16) const; + }; + + /** + * Parse GCS filter from REQUEST_SYNC payload + * @param payload REQUEST_SYNC payload (TLV encoded) + * @param len Payload length + * @param outFilter Output filter structure + * @return true if filter was successfully parsed + */ + bool parseGCSFilter(const uint8_t* payload, size_t len, GCSFilter& outFilter); + + /** + * Find or configure the #mesh channel + * Called during begin() to set up the channel for relaying + */ + void configureMeshChannel(); + + /** + * Check if a MeshCore channel matches the #mesh channel by key + * @param channel Channel to check + * @return true if it's the #mesh channel + */ + bool isMeshChannel(const mesh::GroupChannel& channel) const; + + /** + * Add a message to the history cache + * @param msg Message to cache + */ + void addToMessageHistory(const DogechatMessage& msg); + + /** + * Handle REQUEST_SYNC by sending cached messages + * @param msg The REQUEST_SYNC message + */ + void handleRequestSync(const DogechatMessage& msg); + + // Peer nickname cache (populated from ANNOUNCE messages) + struct PeerInfo { + uint64_t peerId; + char nickname[16]; // 13 chars + null + padding + uint32_t timestamp; // millis() when last seen + bool valid; + }; + static const size_t PEER_CACHE_SIZE = 32; + PeerInfo _peerCache[PEER_CACHE_SIZE]; + + // Fragment reassembly buffers for long messages + // Dogechat fragments messages >245 bytes into multiple FRAGMENT messages + struct FragmentBuffer { + uint64_t senderId; + uint8_t fragmentId; + uint8_t totalFragments; + uint8_t receivedMask; // Bitmask of received fragments (up to 8 fragments) + uint8_t data[2048]; // Reassembly buffer + size_t dataLen; + uint32_t startTime; // millis() when first fragment received + bool active; + }; + static const size_t MAX_FRAGMENT_BUFFERS = 4; + static const uint32_t FRAGMENT_TIMEOUT_MS = 10000; // 10 second timeout + FragmentBuffer _fragmentBuffers[MAX_FRAGMENT_BUFFERS]; + + /** + * Handle incoming fragment message + * Reassembles multi-fragment messages and processes when complete + */ + void handleFragment(const DogechatMessage& msg); + + /** + * Derive Dogechat peer ID from Meshcore identity + * Uses first 8 bytes of public key + */ + uint64_t derivePeerId(const mesh::LocalIdentity& identity); + + /** + * Derive Noise public key (Curve25519) from Ed25519 public key + * Uses standard Edwards→Montgomery conversion + */ + void deriveNoisePublicKey(const uint8_t* ed25519PubKey, uint8_t* curve25519PubKey); + + /** + * Send peer announcement to connected Dogechat clients + */ + void sendPeerAnnouncement(); + + /** + * Sign a Dogechat message with the companion's Ed25519 identity + * Uses Dogechat protocol signing rules (TTL=0, PKCS#7 padding) + */ + void signMessage(DogechatMessage& msg); + + /** + * Process incoming Dogechat message from BLE + */ + void processDogechatMessage(const DogechatMessage& msg); + + /** + * Relay Dogechat channel message to Meshcore mesh + * @param msg Original Dogechat message + * @param channelName Channel name (e.g., "#mesh") + * @param senderNick Sender's nickname from Dogechat + * @param text Message content + */ + void relayChannelMessageToMesh(const DogechatMessage& msg, const char* channelName, + const char* senderNick, const char* text); + + /** + * Relay Dogechat DM to Meshcore mesh + */ + void relayDirectMessageToMesh(const DogechatMessage& msg, const char* text); + + /** + * Broadcast Dogechat message to connected BLE clients + */ + void broadcastToDogechat(const DogechatMessage& msg); + + /** + * Get current time in milliseconds (for Dogechat timestamps) + * Returns synchronized time if available, otherwise falls back to RTC or millis() + */ + uint64_t getCurrentTimeMs(); + + /** + * Synchronize local time from a received Dogechat packet timestamp + * This is critical for proper operation - Android rejects announces with stale timestamps + */ + void syncTimeFromPacket(uint64_t packetTimestamp); + + /** + * Parse ANNOUNCE message TLV to extract nickname + * @param payload TLV-encoded payload + * @param len Payload length + * @param nickname OUT: Nickname if found + * @param nickLen Capacity of nickname buffer + * @return true if nickname was found + */ + bool parseAnnounceTLV(const uint8_t* payload, size_t len, char* nickname, size_t nickLen); + + /** + * Cache or update a peer's nickname + * @param peerId Peer's 64-bit ID + * @param nickname Peer's nickname + */ + void cachePeer(uint64_t peerId, const char* nickname); + + /** + * Look up a cached peer nickname + * @param peerId Peer's 64-bit ID + * @return Nickname or nullptr if not found + */ + const char* lookupPeerNickname(uint64_t peerId); + + /** + * Send a single message part to the mesh (immediate, no delay) + * @param senderNick Sender nickname with emoji prefix + * @param text Message text (may include part indicator) + */ + void sendSingleMessageToMesh(const char* senderNick, const char* text); + + // Pending message parts queue for reliable multi-part message delivery + // Instead of using mesh's delayed transmission (which can fail silently when pool is exhausted), + // we queue parts here and send them one at a time with timer-based delays in loop() + struct PendingPart { + char senderNick[68]; // Includes emoji prefix + char text[180]; // Part text with "[X/Y] " indicator + bool valid; + }; + static const size_t MAX_PENDING_PARTS = 8; + static const uint32_t PART_SEND_DELAY_MS = 5000; // Delay between parts + PendingPart _pendingParts[MAX_PENDING_PARTS]; + size_t _pendingPartsHead; // Next part to send + size_t _pendingPartsTail; // Next slot to queue into + uint32_t _lastPartSentTime; // millis() when last part was sent + + /** + * Queue a message part for delayed sending + * @param senderNick Sender nickname with emoji prefix + * @param text Message text (may include part indicator) + * @return true if queued successfully + */ + bool queueMessagePart(const char* senderNick, const char* text); + + /** + * Process pending message parts queue (called from loop()) + */ + void processPendingParts(); + + /** + * Parse DogechatMessage TLV payload structure + * + * The payload contains: + * [flags:1][timestamp:8][idLen:1][id:N][senderLen:1][sender:N] + * [contentLen:2][content:N]...[channelLen:1][channel:N if hasChannel] + * + * @param payload Message payload (TLV encoded) + * @param payloadLen Payload length + * @param senderNick OUT: Sender's nickname + * @param senderNickLen Capacity of senderNick buffer + * @param content OUT: Message content (text) + * @param contentLen Capacity of content buffer + * @param channelName OUT: Channel name (with #), empty if DM + * @param channelNameLen Capacity of channelName buffer + * @return true if successfully parsed + */ + bool parseDogechatMessageTLV(const uint8_t* payload, size_t payloadLen, + char* senderNick, size_t senderNickLen, + char* content, size_t contentLen, + char* channelName, size_t channelNameLen); +}; diff --git a/src/helpers/dogechat/DogechatProtocol.cpp b/src/helpers/dogechat/DogechatProtocol.cpp new file mode 100644 index 000000000..5128e0a57 --- /dev/null +++ b/src/helpers/dogechat/DogechatProtocol.cpp @@ -0,0 +1,554 @@ +#include "DogechatProtocol.h" +#include "../../Utils.h" + +// Platform-specific miniz includes for DEFLATE decompression +#if defined(ESP32) + #include // For Serial debug output + // Use ESP-IDF's ROM miniz for raw deflate decompression + // tinfl_decompress_mem_to_mem() is available in ESP32 ROM + extern "C" { + #include "rom/miniz.h" + } + #define DOGECHAT_HAS_DECOMPRESSION 1 +#elif defined(NRF52_PLATFORM) + // Use portable miniz library for NRF52 + #include "../miniz/miniz_tinfl.h" + #define DOGECHAT_HAS_DECOMPRESSION 1 +#else + // No decompression support on other platforms + #define DOGECHAT_HAS_DECOMPRESSION 0 +#endif + +// ============================================================================ +// DogechatDuplicateCache +// ============================================================================ + +DogechatDuplicateCache::DogechatDuplicateCache() : currentIndex(0) { + clear(); +} + +void DogechatDuplicateCache::clear() { + for (size_t i = 0; i < DOGECHAT_DUPLICATE_CACHE_SIZE; i++) { + cache[i].valid = false; + cache[i].hash = 0; + cache[i].timestamp = 0; + } + currentIndex = 0; +} + +uint32_t DogechatDuplicateCache::calculateHash(const DogechatMessage& msg) const { + // FNV-1a hash algorithm + uint32_t hash = 2166136261u; // FNV offset basis + + // Hash sender ID + for (int i = 0; i < DOGECHAT_SENDER_ID_SIZE; i++) { + hash ^= msg.senderId[i]; + hash *= 16777619u; // FNV prime + } + + // Hash timestamp (lower 32 bits, in seconds for tolerance) + uint32_t ts_sec = static_cast(msg.timestamp / 1000); + hash ^= (ts_sec & 0xFF); + hash *= 16777619u; + hash ^= ((ts_sec >> 8) & 0xFF); + hash *= 16777619u; + hash ^= ((ts_sec >> 16) & 0xFF); + hash *= 16777619u; + hash ^= ((ts_sec >> 24) & 0xFF); + hash *= 16777619u; + + // Hash message type + hash ^= msg.type; + hash *= 16777619u; + + // Hash payload length + hash ^= (msg.payloadLength & 0xFF); + hash *= 16777619u; + hash ^= ((msg.payloadLength >> 8) & 0xFF); + hash *= 16777619u; + + // Hash first 16 bytes of payload (if available) + size_t hashLen = msg.payloadLength < 16 ? msg.payloadLength : 16; + for (size_t i = 0; i < hashLen; i++) { + hash ^= msg.payload[i]; + hash *= 16777619u; + } + + return hash; +} + +bool DogechatDuplicateCache::isDuplicate(const DogechatMessage& msg) { + uint32_t hash = calculateHash(msg); + uint32_t ts_sec = static_cast(msg.timestamp / 1000); + + // Check existing entries + for (size_t i = 0; i < DOGECHAT_DUPLICATE_CACHE_SIZE; i++) { + if (!cache[i].valid) continue; + + if (cache[i].hash == hash) { + // Allow ±5 second timestamp tolerance for duplicates + int32_t timeDiff = static_cast(ts_sec) - static_cast(cache[i].timestamp); + if (timeDiff >= -5 && timeDiff <= 5) { + return true; + } + } + } + + // Not a duplicate - add to cache + addMessage(msg); + return false; +} + +void DogechatDuplicateCache::addMessage(const DogechatMessage& msg) { + cache[currentIndex].hash = calculateHash(msg); + cache[currentIndex].timestamp = static_cast(msg.timestamp / 1000); + cache[currentIndex].valid = true; + + currentIndex = (currentIndex + 1) % DOGECHAT_DUPLICATE_CACHE_SIZE; +} + +// ============================================================================ +// DogechatProtocol - Helper functions +// ============================================================================ + +uint16_t DogechatProtocol::readBE16(const uint8_t* data) { + return (static_cast(data[0]) << 8) | data[1]; +} + +uint64_t DogechatProtocol::readBE64(const uint8_t* data) { + uint64_t result = 0; + for (int i = 0; i < 8; i++) { + result = (result << 8) | data[i]; + } + return result; +} + +void DogechatProtocol::writeBE16(uint8_t* data, uint16_t value) { + data[0] = static_cast((value >> 8) & 0xFF); + data[1] = static_cast(value & 0xFF); +} + +void DogechatProtocol::writeBE64(uint8_t* data, uint64_t value) { + for (int i = 7; i >= 0; i--) { + data[7 - i] = static_cast((value >> (i * 8)) & 0xFF); + } +} + +// ============================================================================ +// DogechatProtocol - Parsing and Serialization +// ============================================================================ + +bool DogechatProtocol::parseMessage(const uint8_t* data, size_t length, DogechatMessage& msg) { + if (length < DOGECHAT_HEADER_SIZE) { + return false; + } + + size_t offset = 0; + + // Parse header + msg.version = data[offset++]; + msg.type = data[offset++]; + msg.ttl = data[offset++]; + msg.timestamp = readBE64(&data[offset]); + offset += 8; + msg.flags = data[offset++]; + msg.payloadLength = readBE16(&data[offset]); + offset += 2; + + // Validate version + if (msg.version != DOGECHAT_VERSION) { + return false; + } + + // Validate payload length + if (msg.payloadLength > DOGECHAT_MAX_PAYLOAD_SIZE) { + return false; + } + + // Calculate expected message size + size_t expectedSize = DOGECHAT_HEADER_SIZE + DOGECHAT_SENDER_ID_SIZE; + if (msg.hasRecipient()) { + expectedSize += DOGECHAT_RECIPIENT_ID_SIZE; + } + expectedSize += msg.payloadLength; + if (msg.hasSignature()) { + expectedSize += DOGECHAT_SIGNATURE_SIZE; + } + + if (length < expectedSize) { + return false; + } + + // Parse sender ID + memcpy(msg.senderId, &data[offset], DOGECHAT_SENDER_ID_SIZE); + offset += DOGECHAT_SENDER_ID_SIZE; + + // Parse recipient ID (if present) + memset(msg.recipientId, 0, DOGECHAT_RECIPIENT_ID_SIZE); + if (msg.hasRecipient()) { + memcpy(msg.recipientId, &data[offset], DOGECHAT_RECIPIENT_ID_SIZE); + offset += DOGECHAT_RECIPIENT_ID_SIZE; + } + + // Parse payload + memset(msg.payload, 0, DOGECHAT_MAX_PAYLOAD_SIZE); + uint16_t wirePayloadLength = msg.payloadLength; // Save original wire length + if (wirePayloadLength > 0) { + // Check if payload is compressed + if (msg.isCompressed()) { +#if DOGECHAT_HAS_DECOMPRESSION + // Compressed payload format (from Android CompressionUtil.kt): + // - First 2 bytes: original uncompressed size (big-endian) + // - Remaining bytes: raw deflate compressed data + if (wirePayloadLength < 3) { + return false; + } + + // Read original size from first 2 bytes + uint16_t originalSize = (data[offset] << 8) | data[offset + 1]; + const uint8_t* compressedData = &data[offset + 2]; + size_t compressedLen = wirePayloadLength - 2; + + if (originalSize > DOGECHAT_MAX_PAYLOAD_SIZE) { + return false; + } + + // Use streaming tinfl API with heap-allocated decompressor to avoid stack overflow + // tinfl_decompress_mem_to_mem allocates ~10KB decompressor internally on stack + tinfl_decompressor* decomp = (tinfl_decompressor*)malloc(sizeof(tinfl_decompressor)); + uint8_t* decompBuffer = (uint8_t*)malloc(DOGECHAT_MAX_PAYLOAD_SIZE); + + if (decomp == nullptr || decompBuffer == nullptr) { + if (decomp) free(decomp); + if (decompBuffer) free(decompBuffer); + return false; + } + + tinfl_init(decomp); + + size_t inBytes = compressedLen; + size_t outBytes = DOGECHAT_MAX_PAYLOAD_SIZE; + + // TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF is required for linear output buffer + // Without it, tinfl expects a ring buffer dictionary + const int linearBufFlag = TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF; + + // Try raw deflate first (Android uses raw deflate, not zlib) + tinfl_status status = tinfl_decompress( + decomp, + compressedData, // Input + &inBytes, // Input size (updated) + decompBuffer, // Output buffer start + decompBuffer, // Output write position + &outBytes, // Output size (updated) + linearBufFlag // Linear buffer, raw deflate + ); + + // If raw deflate failed, try with zlib header + if (status != TINFL_STATUS_DONE) { + tinfl_init(decomp); + inBytes = compressedLen; + outBytes = DOGECHAT_MAX_PAYLOAD_SIZE; + status = tinfl_decompress( + decomp, + compressedData, + &inBytes, + decompBuffer, + decompBuffer, + &outBytes, + linearBufFlag | TINFL_FLAG_PARSE_ZLIB_HEADER + ); + } + + if (status != TINFL_STATUS_DONE) { + free(decomp); + free(decompBuffer); + return false; + } + + // Copy decompressed data to message payload + size_t decompressedLen = outBytes; + memcpy(msg.payload, decompBuffer, decompressedLen); + free(decomp); + free(decompBuffer); + + msg.payloadLength = static_cast(decompressedLen); + msg.flags &= ~DOGECHAT_FLAG_IS_COMPRESSED; // Clear compressed flag +#else + // Platforms without decompression support: just copy raw payload + memcpy(msg.payload, &data[offset], wirePayloadLength); +#endif + } else { + // Uncompressed payload - direct copy + memcpy(msg.payload, &data[offset], wirePayloadLength); + } + offset += wirePayloadLength; + } + + // Parse signature (if present) + memset(msg.signature, 0, DOGECHAT_SIGNATURE_SIZE); + if (msg.hasSignature()) { + memcpy(msg.signature, &data[offset], DOGECHAT_SIGNATURE_SIZE); + offset += DOGECHAT_SIGNATURE_SIZE; + } + + return true; +} + +size_t DogechatProtocol::serializeMessage(const DogechatMessage& msg, uint8_t* buffer, size_t maxLength) { + size_t requiredSize = getMessageSize(msg); + if (maxLength < requiredSize) { + return 0; + } + + size_t offset = 0; + + // Write header + buffer[offset++] = msg.version; + buffer[offset++] = msg.type; + buffer[offset++] = msg.ttl; + writeBE64(&buffer[offset], msg.timestamp); + offset += 8; + buffer[offset++] = msg.flags; + writeBE16(&buffer[offset], msg.payloadLength); + offset += 2; + + // Write sender ID + memcpy(&buffer[offset], msg.senderId, DOGECHAT_SENDER_ID_SIZE); + offset += DOGECHAT_SENDER_ID_SIZE; + + // Write recipient ID (if present) + if (msg.hasRecipient()) { + memcpy(&buffer[offset], msg.recipientId, DOGECHAT_RECIPIENT_ID_SIZE); + offset += DOGECHAT_RECIPIENT_ID_SIZE; + } + + // Write payload + if (msg.payloadLength > 0) { + memcpy(&buffer[offset], msg.payload, msg.payloadLength); + offset += msg.payloadLength; + } + + // Write signature (if present) + if (msg.hasSignature()) { + memcpy(&buffer[offset], msg.signature, DOGECHAT_SIGNATURE_SIZE); + offset += DOGECHAT_SIGNATURE_SIZE; + } + + return offset; +} + +bool DogechatProtocol::validateMessage(const DogechatMessage& msg) { + // Check version + if (msg.version != DOGECHAT_VERSION) { + return false; + } + + // Check type is valid + switch (msg.type) { + case DOGECHAT_MSG_ANNOUNCE: + case DOGECHAT_MSG_MESSAGE: + case DOGECHAT_MSG_LEAVE: + case DOGECHAT_MSG_IDENTITY: + case DOGECHAT_MSG_CHANNEL: + case DOGECHAT_MSG_PING: + case DOGECHAT_MSG_PONG: + case DOGECHAT_MSG_NOISE_HANDSHAKE: + case DOGECHAT_MSG_NOISE_ENCRYPTED: + case DOGECHAT_MSG_FRAGMENT_NEW: + case DOGECHAT_MSG_REQUEST_SYNC: + case DOGECHAT_MSG_FILE_TRANSFER: + case DOGECHAT_MSG_FRAGMENT: + break; + default: + return false; + } + + // Check payload length + if (msg.payloadLength > DOGECHAT_MAX_PAYLOAD_SIZE) { + return false; + } + + // Check sender ID is non-zero + bool senderNonZero = false; + for (int i = 0; i < DOGECHAT_SENDER_ID_SIZE; i++) { + if (msg.senderId[i] != 0) { + senderNonZero = true; + break; + } + } + if (!senderNonZero) { + return false; + } + + return true; +} + +size_t DogechatProtocol::getMessageSize(const DogechatMessage& msg) { + size_t size = DOGECHAT_HEADER_SIZE + DOGECHAT_SENDER_ID_SIZE; + + if (msg.hasRecipient()) { + size += DOGECHAT_RECIPIENT_ID_SIZE; + } + + size += msg.payloadLength; + + if (msg.hasSignature()) { + size += DOGECHAT_SIGNATURE_SIZE; + } + + return size; +} + +void DogechatProtocol::computePacketId(const DogechatMessage& msg, uint8_t* outId16) { + // Compute packet ID matching Android Dogechat: + // SHA-256(type | senderId | timestamp_BE | payload)[0:16] + // + // This creates a deterministic unique ID for each message based on its content. + // Used by GCS filter to detect which messages the requester already has. + + // Build the data to hash: type(1) + senderId(8) + timestamp(8 BE) + payload + uint8_t hashInput[1 + DOGECHAT_SENDER_ID_SIZE + 8 + DOGECHAT_MAX_PAYLOAD_SIZE]; + size_t offset = 0; + + // Type (1 byte) + hashInput[offset++] = msg.type; + + // Sender ID (8 bytes, as stored - little endian) + memcpy(&hashInput[offset], msg.senderId, DOGECHAT_SENDER_ID_SIZE); + offset += DOGECHAT_SENDER_ID_SIZE; + + // Timestamp (8 bytes, big-endian) + writeBE64(&hashInput[offset], msg.timestamp); + offset += 8; + + // Payload + if (msg.payloadLength > 0) { + memcpy(&hashInput[offset], msg.payload, msg.payloadLength); + offset += msg.payloadLength; + } + + // Compute SHA-256 and truncate to 16 bytes + uint8_t fullHash[32]; + mesh::Utils::sha256(fullHash, 32, hashInput, static_cast(offset)); + + // Copy first 16 bytes as the packet ID + memcpy(outId16, fullHash, 16); +} + +// ============================================================================ +// DogechatProtocol - Message Creation +// ============================================================================ + +void DogechatProtocol::createAnnounce(DogechatMessage& msg, uint64_t senderId, const char* nickname, + const uint8_t* noisePublicKey, const uint8_t* signingPublicKey, + uint64_t timestamp, uint8_t ttl) { + msg.version = DOGECHAT_VERSION; + msg.type = DOGECHAT_MSG_ANNOUNCE; + msg.ttl = ttl; + msg.timestamp = timestamp; + msg.flags = 0; // No recipient, no signature for basic announce + msg.setSenderId64(senderId); + + // Build TLV payload + size_t offset = 0; + + // Add nickname TLV (0x01) + // NOTE: Nickname is limited to 13 bytes to ensure signed announce packet fits within + // BLE MTU of 169 bytes. Total: header(13) + sender(8) + payload(84 max) + sig(64) = 169 + // Payload: nick_tlv(2+13=15) + noise_tlv(34) + ed25519_tlv(34) = 83 bytes + if (nickname != nullptr && nickname[0] != '\0') { + size_t nickLen = strlen(nickname); + if (nickLen > 13) nickLen = 13; // Limit to ensure packet fits in 169-byte MTU + + if (offset + 2 + nickLen <= DOGECHAT_MAX_PAYLOAD_SIZE) { + msg.payload[offset++] = DOGECHAT_TLV_NICKNAME; + msg.payload[offset++] = static_cast(nickLen); + memcpy(&msg.payload[offset], nickname, nickLen); + offset += nickLen; + } + } + + // Add Noise public key TLV (0x02) - Curve25519 for Noise protocol + if (noisePublicKey != nullptr) { + if (offset + 2 + 32 <= DOGECHAT_MAX_PAYLOAD_SIZE) { + msg.payload[offset++] = DOGECHAT_TLV_NOISE_PUBKEY; + msg.payload[offset++] = 32; + memcpy(&msg.payload[offset], noisePublicKey, 32); + offset += 32; + } + } + + // Add Ed25519 signing public key TLV (0x03) + if (signingPublicKey != nullptr) { + if (offset + 2 + 32 <= DOGECHAT_MAX_PAYLOAD_SIZE) { + msg.payload[offset++] = DOGECHAT_TLV_ED25519_PUBKEY; + msg.payload[offset++] = 32; + memcpy(&msg.payload[offset], signingPublicKey, 32); + offset += 32; + } + } + + msg.payloadLength = static_cast(offset); +} + +void DogechatProtocol::createTextMessage(DogechatMessage& msg, uint64_t senderId, uint64_t recipientId, + const char* channelName, const char* text, size_t textLen, + uint64_t timestamp, uint8_t ttl) { + msg.version = DOGECHAT_VERSION; + msg.type = DOGECHAT_MSG_MESSAGE; + msg.ttl = ttl; + msg.timestamp = timestamp; + msg.setSenderId64(senderId); + + size_t offset = 0; + + if (recipientId != 0) { + // Direct message + msg.flags = DOGECHAT_FLAG_HAS_RECIPIENT; + msg.setRecipientId64(recipientId); + + // Payload is just the text + if (textLen > DOGECHAT_MAX_PAYLOAD_SIZE) { + textLen = DOGECHAT_MAX_PAYLOAD_SIZE; + } + memcpy(msg.payload, text, textLen); + offset = textLen; + } else if (channelName != nullptr && channelName[0] != '\0') { + // Channel message - format: "#channel:text" + msg.flags = 0; // No recipient + memset(msg.recipientId, 0, DOGECHAT_RECIPIENT_ID_SIZE); + + // Build payload: #channelname\0text + size_t channelLen = strlen(channelName); + + // Add # prefix and channel name + if (offset < DOGECHAT_MAX_PAYLOAD_SIZE) { + msg.payload[offset++] = '#'; + } + for (size_t i = 0; i < channelLen && offset < DOGECHAT_MAX_PAYLOAD_SIZE; i++) { + msg.payload[offset++] = channelName[i]; + } + + // Add separator (colon) + if (offset < DOGECHAT_MAX_PAYLOAD_SIZE) { + msg.payload[offset++] = ':'; + } + + // Add text + for (size_t i = 0; i < textLen && offset < DOGECHAT_MAX_PAYLOAD_SIZE; i++) { + msg.payload[offset++] = text[i]; + } + } else { + // No recipient and no channel - invalid, but handle gracefully + msg.flags = 0; + memset(msg.recipientId, 0, DOGECHAT_RECIPIENT_ID_SIZE); + if (textLen > DOGECHAT_MAX_PAYLOAD_SIZE) { + textLen = DOGECHAT_MAX_PAYLOAD_SIZE; + } + memcpy(msg.payload, text, textLen); + offset = textLen; + } + + msg.payloadLength = static_cast(offset); +} diff --git a/src/helpers/dogechat/DogechatProtocol.h b/src/helpers/dogechat/DogechatProtocol.h new file mode 100644 index 000000000..be59b7b03 --- /dev/null +++ b/src/helpers/dogechat/DogechatProtocol.h @@ -0,0 +1,238 @@ +#pragma once + +#include +#include +#include + +// Dogechat Protocol Constants +#define DOGECHAT_HEADER_SIZE 14 // version(1) + type(1) + ttl(1) + timestamp(8) + flags(1) + payloadLength(2) +#define DOGECHAT_SIGNATURE_SIZE 64 // Ed25519 signature +#define DOGECHAT_MAX_WIRE_PAYLOAD_SIZE 245 // Max payload size on wire (compressed/padded) +#define DOGECHAT_MAX_PAYLOAD_SIZE 512 // Max decompressed payload size (reduced from 1024 to prevent stack overflow) +#define DOGECHAT_VERSION 1 +#define DOGECHAT_SENDER_ID_SIZE 8 +#define DOGECHAT_RECIPIENT_ID_SIZE 8 + +// Maximum message size on wire: header(13) + sender(8) + recipient(8) + payload(245) + signature(64) = 338 bytes +#define DOGECHAT_MAX_MESSAGE_SIZE (DOGECHAT_HEADER_SIZE + DOGECHAT_SENDER_ID_SIZE + DOGECHAT_RECIPIENT_ID_SIZE + DOGECHAT_MAX_WIRE_PAYLOAD_SIZE + DOGECHAT_SIGNATURE_SIZE) + +// BLE Service UUIDs +#define DOGECHAT_SERVICE_UUID "F47B5E2D-4A9E-4C5A-9B3F-8E1D2C3A4B5C" +#define DOGECHAT_CHARACTERISTIC_UUID "A1B2C3D4-E5F6-4A5B-8C9D-0E1F2A3B4C5D" + +// Duplicate cache configuration +#define DOGECHAT_DUPLICATE_CACHE_SIZE 100 +#define DOGECHAT_DUPLICATE_TIME_WINDOW_MS 300000 // 5 minutes + +// Dogechat Message Types +enum DogechatMessageType : uint8_t { + DOGECHAT_MSG_ANNOUNCE = 0x01, + DOGECHAT_MSG_MESSAGE = 0x02, + DOGECHAT_MSG_LEAVE = 0x03, + DOGECHAT_MSG_IDENTITY = 0x04, + DOGECHAT_MSG_CHANNEL = 0x05, + DOGECHAT_MSG_PING = 0x06, + DOGECHAT_MSG_PONG = 0x07, + DOGECHAT_MSG_NOISE_HANDSHAKE = 0x10, + DOGECHAT_MSG_NOISE_ENCRYPTED = 0x11, + DOGECHAT_MSG_FRAGMENT_NEW = 0x20, + DOGECHAT_MSG_REQUEST_SYNC = 0x21, + DOGECHAT_MSG_FILE_TRANSFER = 0x22, + DOGECHAT_MSG_FRAGMENT = 0xFF +}; + +// Dogechat Protocol Flags +#define DOGECHAT_FLAG_HAS_RECIPIENT 0x01 +#define DOGECHAT_FLAG_HAS_SIGNATURE 0x02 +#define DOGECHAT_FLAG_IS_COMPRESSED 0x04 + +// Announce payload TLV types +#define DOGECHAT_TLV_NICKNAME 0x01 +#define DOGECHAT_TLV_NOISE_PUBKEY 0x02 +#define DOGECHAT_TLV_ED25519_PUBKEY 0x03 + +/** + * Dogechat Protocol Message Structure + * Matches iOS BinaryProtocol format for compatibility + */ +struct DogechatMessage { + uint8_t version; + uint8_t type; + uint8_t ttl; + uint64_t timestamp; // milliseconds since epoch + uint8_t flags; + uint16_t payloadLength; + uint8_t senderId[DOGECHAT_SENDER_ID_SIZE]; + uint8_t recipientId[DOGECHAT_RECIPIENT_ID_SIZE]; + uint8_t payload[DOGECHAT_MAX_PAYLOAD_SIZE]; + uint8_t signature[DOGECHAT_SIGNATURE_SIZE]; + + DogechatMessage() : version(DOGECHAT_VERSION), type(0), ttl(0), timestamp(0), flags(0), payloadLength(0) { + memset(senderId, 0, sizeof(senderId)); + memset(recipientId, 0, sizeof(recipientId)); + memset(payload, 0, sizeof(payload)); + memset(signature, 0, sizeof(signature)); + } + + bool hasRecipient() const { return (flags & DOGECHAT_FLAG_HAS_RECIPIENT) != 0; } + bool hasSignature() const { return (flags & DOGECHAT_FLAG_HAS_SIGNATURE) != 0; } + bool isCompressed() const { return (flags & DOGECHAT_FLAG_IS_COMPRESSED) != 0; } + + void setHasRecipient(bool val) { if (val) flags |= DOGECHAT_FLAG_HAS_RECIPIENT; else flags &= ~DOGECHAT_FLAG_HAS_RECIPIENT; } + void setHasSignature(bool val) { if (val) flags |= DOGECHAT_FLAG_HAS_SIGNATURE; else flags &= ~DOGECHAT_FLAG_HAS_SIGNATURE; } + + // Get sender ID as 64-bit integer (little-endian) + uint64_t getSenderId64() const { + uint64_t id = 0; + for (int i = 0; i < 8; i++) { + id |= (static_cast(senderId[i]) << (i * 8)); + } + return id; + } + + // Set sender ID from 64-bit integer (little-endian) + void setSenderId64(uint64_t id) { + for (int i = 0; i < 8; i++) { + senderId[i] = static_cast((id >> (i * 8)) & 0xFF); + } + } + + // Get recipient ID as 64-bit integer + uint64_t getRecipientId64() const { + uint64_t id = 0; + for (int i = 0; i < 8; i++) { + id |= (static_cast(recipientId[i]) << (i * 8)); + } + return id; + } + + // Set recipient ID from 64-bit integer + void setRecipientId64(uint64_t id) { + for (int i = 0; i < 8; i++) { + recipientId[i] = static_cast((id >> (i * 8)) & 0xFF); + } + } +}; + +/** + * Duplicate Message Cache + * Prevents message loops by tracking recently seen messages + */ +class DogechatDuplicateCache { +public: + DogechatDuplicateCache(); + + // Check if message is a duplicate (and add if not) + bool isDuplicate(const DogechatMessage& msg); + + // Explicitly add a message to the cache + void addMessage(const DogechatMessage& msg); + + // Clear the cache + void clear(); + +private: + struct CacheEntry { + uint32_t hash; + uint32_t timestamp; // seconds + bool valid; + + CacheEntry() : hash(0), timestamp(0), valid(false) {} + }; + + CacheEntry cache[DOGECHAT_DUPLICATE_CACHE_SIZE]; + size_t currentIndex; + + // FNV-1a hash + uint32_t calculateHash(const DogechatMessage& msg) const; +}; + +/** + * Protocol parsing and serialization + */ +class DogechatProtocol { +public: + /** + * Parse a binary buffer into a DogechatMessage + * @param data Input buffer + * @param length Buffer length + * @param msg Output message + * @return true if parsing succeeded + */ + static bool parseMessage(const uint8_t* data, size_t length, DogechatMessage& msg); + + /** + * Serialize a DogechatMessage to a binary buffer + * @param msg Input message + * @param buffer Output buffer + * @param maxLength Buffer capacity + * @return Number of bytes written, or 0 on failure + */ + static size_t serializeMessage(const DogechatMessage& msg, uint8_t* buffer, size_t maxLength); + + /** + * Validate a DogechatMessage + * @param msg Message to validate + * @return true if message is valid + */ + static bool validateMessage(const DogechatMessage& msg); + + /** + * Get the serialized size of a message + * @param msg Message to measure + * @return Size in bytes + */ + static size_t getMessageSize(const DogechatMessage& msg); + + /** + * Compute the deterministic packet ID for a message + * This matches the Android Dogechat packet ID computation: + * SHA-256(type | senderId | timestamp_BE | payload)[0:16] + * + * @param msg Message to compute ID for + * @param outId16 Output buffer (16 bytes) for the packet ID + */ + static void computePacketId(const DogechatMessage& msg, uint8_t* outId16); + + /** + * Create an ANNOUNCE message + * @param msg Output message + * @param senderId Sender's 64-bit ID + * @param nickname Sender's nickname (null-terminated) + * @param noisePublicKey Curve25519 public key for Noise protocol (32 bytes, can be NULL) + * @param signingPublicKey Ed25519 public key for signatures (32 bytes, can be NULL) + * @param timestamp Current time in milliseconds + * @param ttl Time-to-live + */ + static void createAnnounce(DogechatMessage& msg, uint64_t senderId, const char* nickname, + const uint8_t* noisePublicKey, const uint8_t* signingPublicKey, + uint64_t timestamp, uint8_t ttl); + + /** + * Create a MESSAGE (text message) + * @param msg Output message + * @param senderId Sender's 64-bit ID + * @param recipientId Recipient's 64-bit ID (0 for channel message) + * @param channelName Channel name (for channel messages, without #) + * @param text Message text + * @param textLen Text length + * @param timestamp Current time in milliseconds + * @param ttl Time-to-live + */ + static void createTextMessage(DogechatMessage& msg, uint64_t senderId, uint64_t recipientId, + const char* channelName, const char* text, size_t textLen, + uint64_t timestamp, uint8_t ttl); + +private: + // Read big-endian uint16 + static uint16_t readBE16(const uint8_t* data); + + // Read big-endian uint64 + static uint64_t readBE64(const uint8_t* data); + + // Write big-endian uint16 + static void writeBE16(uint8_t* data, uint16_t value); + + // Write big-endian uint64 + static void writeBE64(uint8_t* data, uint64_t value); +}; diff --git a/src/helpers/esp32/SerialBLEInterface.cpp b/src/helpers/esp32/SerialBLEInterface.cpp index 7ec937238..e5a56e2c2 100644 --- a/src/helpers/esp32/SerialBLEInterface.cpp +++ b/src/helpers/esp32/SerialBLEInterface.cpp @@ -1,5 +1,9 @@ #include "SerialBLEInterface.h" +#ifdef ENABLE_DOGECHAT +#include "../dogechat/DogechatBLEService.h" +#endif + // See the following for generating UUIDs: // https://www.uuidgenerator.net/ @@ -97,6 +101,12 @@ void SerialBLEInterface::onDisconnect(BLEServer* pServer) { // loop() will detect this on next loop, and set deviceConnected to false } + +#ifdef ENABLE_DOGECHAT + if (_dogechatService != nullptr) { + _dogechatService->onServerDisconnect(); + } +#endif } // -------- BLECharacteristicCallbacks methods diff --git a/src/helpers/esp32/SerialBLEInterface.h b/src/helpers/esp32/SerialBLEInterface.h index 29ad897ae..fe0252f72 100644 --- a/src/helpers/esp32/SerialBLEInterface.h +++ b/src/helpers/esp32/SerialBLEInterface.h @@ -6,6 +6,10 @@ #include #include +#ifdef ENABLE_DOGECHAT +class DogechatBLEService; +#endif + class SerialBLEInterface : public BaseSerialInterface, BLESecurityCallbacks, BLEServerCallbacks, BLECharacteristicCallbacks { BLEServer *pServer; BLEService *pService; @@ -73,6 +77,15 @@ class SerialBLEInterface : public BaseSerialInterface, BLESecurityCallbacks, BLE bool isWriteBusy() const override; size_t writeFrame(const uint8_t src[], size_t len) override; size_t checkRecvFrame(uint8_t dest[]) override; + + // Expose BLE server for adding additional services (e.g., Dogechat) + BLEServer* getBLEServer() { return pServer; } + +#ifdef ENABLE_DOGECHAT + void setDogechatService(DogechatBLEService* service) { _dogechatService = service; } +private: + DogechatBLEService* _dogechatService = nullptr; +#endif }; #if BLE_DEBUG_LOGGING && ARDUINO diff --git a/src/helpers/miniz/miniz.c b/src/helpers/miniz/miniz.c new file mode 100644 index 000000000..d07b98d71 --- /dev/null +++ b/src/helpers/miniz/miniz.c @@ -0,0 +1,646 @@ +/************************************************************************** + * + * Copyright 2013-2014 RAD Game Tools and Valve Software + * Copyright 2010-2014 Rich Geldreich and Tenacious Software LLC + * All Rights Reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + **************************************************************************/ + +#include "miniz.h" + +typedef unsigned char mz_validate_uint16[sizeof(mz_uint16) == 2 ? 1 : -1]; +typedef unsigned char mz_validate_uint32[sizeof(mz_uint32) == 4 ? 1 : -1]; +typedef unsigned char mz_validate_uint64[sizeof(mz_uint64) == 8 ? 1 : -1]; + +#ifdef __cplusplus +extern "C" +{ +#endif + + /* ------------------- zlib-style API's */ + + mz_ulong mz_adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len) + { + mz_uint32 i, s1 = (mz_uint32)(adler & 0xffff), s2 = (mz_uint32)(adler >> 16); + size_t block_len = buf_len % 5552; + if (!ptr) + return MZ_ADLER32_INIT; + while (buf_len) + { + for (i = 0; i + 7 < block_len; i += 8, ptr += 8) + { + s1 += ptr[0], s2 += s1; + s1 += ptr[1], s2 += s1; + s1 += ptr[2], s2 += s1; + s1 += ptr[3], s2 += s1; + s1 += ptr[4], s2 += s1; + s1 += ptr[5], s2 += s1; + s1 += ptr[6], s2 += s1; + s1 += ptr[7], s2 += s1; + } + for (; i < block_len; ++i) + s1 += *ptr++, s2 += s1; + s1 %= 65521U, s2 %= 65521U; + buf_len -= block_len; + block_len = 5552; + } + return (s2 << 16) + s1; + } + +/* Karl Malbrain's compact CRC-32. See "A compact CCITT crc16 and crc32 C implementation that balances processor cache usage against speed": http://www.geocities.com/malbrain/ */ +#if 0 + mz_ulong mz_crc32(mz_ulong crc, const mz_uint8 *ptr, size_t buf_len) + { + static const mz_uint32 s_crc32[16] = { 0, 0x1db71064, 0x3b6e20c8, 0x26d930ac, 0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c, + 0xedb88320, 0xf00f9344, 0xd6d6a3e8, 0xcb61b38c, 0x9b64c2b0, 0x86d3d2d4, 0xa00ae278, 0xbdbdf21c }; + mz_uint32 crcu32 = (mz_uint32)crc; + if (!ptr) + return MZ_CRC32_INIT; + crcu32 = ~crcu32; + while (buf_len--) + { + mz_uint8 b = *ptr++; + crcu32 = (crcu32 >> 4) ^ s_crc32[(crcu32 & 0xF) ^ (b & 0xF)]; + crcu32 = (crcu32 >> 4) ^ s_crc32[(crcu32 & 0xF) ^ (b >> 4)]; + } + return ~crcu32; + } +#elif defined(USE_EXTERNAL_MZCRC) +/* If USE_EXTERNAL_CRC is defined, an external module will export the + * mz_crc32() symbol for us to use, e.g. an SSE-accelerated version. + * Depending on the impl, it may be necessary to ~ the input/output crc values. + */ +mz_ulong mz_crc32(mz_ulong crc, const mz_uint8 *ptr, size_t buf_len); +#else +/* Faster, but larger CPU cache footprint. + */ +mz_ulong mz_crc32(mz_ulong crc, const mz_uint8 *ptr, size_t buf_len) +{ + static const mz_uint32 s_crc_table[256] = { + 0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419, 0x706AF48F, 0xE963A535, + 0x9E6495A3, 0x0EDB8832, 0x79DCB8A4, 0xE0D5E91E, 0x97D2D988, 0x09B64C2B, 0x7EB17CBD, + 0xE7B82D07, 0x90BF1D91, 0x1DB71064, 0x6AB020F2, 0xF3B97148, 0x84BE41DE, 0x1ADAD47D, + 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7, 0x136C9856, 0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, + 0x14015C4F, 0x63066CD9, 0xFA0F3D63, 0x8D080DF5, 0x3B6E20C8, 0x4C69105E, 0xD56041E4, + 0xA2677172, 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B, 0x35B5A8FA, 0x42B2986C, + 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3, 0x45DF5C75, 0xDCD60DCF, 0xABD13D59, 0x26D930AC, + 0x51DE003A, 0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, 0xCFBA9599, 0xB8BDA50F, + 0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924, 0x2F6F7C87, 0x58684C11, 0xC1611DAB, + 0xB6662D3D, 0x76DC4190, 0x01DB7106, 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F, + 0x9FBFE4A5, 0xE8B8D433, 0x7807C9A2, 0x0F00F934, 0x9609A88E, 0xE10E9818, 0x7F6A0DBB, + 0x086D3D2D, 0x91646C97, 0xE6635C01, 0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, + 0x6C0695ED, 0x1B01A57B, 0x8208F4C1, 0xF50FC457, 0x65B0D9C6, 0x12B7E950, 0x8BBEB8EA, + 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3, 0xFBD44C65, 0x4DB26158, 0x3AB551CE, + 0xA3BC0074, 0xD4BB30E2, 0x4ADFA541, 0x3DD895D7, 0xA4D1C46D, 0xD3D6F4FB, 0x4369E96A, + 0x346ED9FC, 0xAD678846, 0xDA60B8D0, 0x44042D73, 0x33031DE5, 0xAA0A4C5F, 0xDD0D7CC9, + 0x5005713C, 0x270241AA, 0xBE0B1010, 0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, + 0xCE61E49F, 0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17, 0x2EB40D81, + 0xB7BD5C3B, 0xC0BA6CAD, 0xEDB88320, 0x9ABFB3B6, 0x03B6E20C, 0x74B1D29A, 0xEAD54739, + 0x9DD277AF, 0x04DB2615, 0x73DC1683, 0xE3630B12, 0x94643B84, 0x0D6D6A3E, 0x7A6A5AA8, + 0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1, 0xF00F9344, 0x8708A3D2, 0x1E01F268, + 0x6906C2FE, 0xF762575D, 0x806567CB, 0x196C3671, 0x6E6B06E7, 0xFED41B76, 0x89D32BE0, + 0x10DA7A5A, 0x67DD4ACC, 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5, 0xD6D6A3E8, + 0xA1D1937E, 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1, 0xA6BC5767, 0x3FB506DD, 0x48B2364B, + 0xD80D2BDA, 0xAF0A1B4C, 0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, 0x316E8EEF, + 0x4669BE79, 0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236, 0xCC0C7795, 0xBB0B4703, + 0x220216B9, 0x5505262F, 0xC5BA3BBE, 0xB2BD0B28, 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, + 0xB5D0CF31, 0x2CD99E8B, 0x5BDEAE1D, 0x9B64C2B0, 0xEC63F226, 0x756AA39C, 0x026D930A, + 0x9C0906A9, 0xEB0E363F, 0x72076785, 0x05005713, 0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, + 0x0CB61B38, 0x92D28E9B, 0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21, 0x86D3D2D4, 0xF1D4E242, + 0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1, 0x18B74777, 0x88085AE6, + 0xFF0F6A70, 0x66063BCA, 0x11010B5C, 0x8F659EFF, 0xF862AE69, 0x616BFFD3, 0x166CCF45, + 0xA00AE278, 0xD70DD2EE, 0x4E048354, 0x3903B3C2, 0xA7672661, 0xD06016F7, 0x4969474D, + 0x3E6E77DB, 0xAED16A4A, 0xD9D65ADC, 0x40DF0B66, 0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, + 0x47B2CF7F, 0x30B5FFE9, 0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605, + 0xCDD70693, 0x54DE5729, 0x23D967BF, 0xB3667A2E, 0xC4614AB8, 0x5D681B02, 0x2A6F2B94, + 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B, 0x2D02EF8D + }; + + mz_uint32 crc32 = (mz_uint32)crc ^ 0xFFFFFFFF; + const mz_uint8 *pByte_buf = (const mz_uint8 *)ptr; + + while (buf_len >= 4) + { + crc32 = (crc32 >> 8) ^ s_crc_table[(crc32 ^ pByte_buf[0]) & 0xFF]; + crc32 = (crc32 >> 8) ^ s_crc_table[(crc32 ^ pByte_buf[1]) & 0xFF]; + crc32 = (crc32 >> 8) ^ s_crc_table[(crc32 ^ pByte_buf[2]) & 0xFF]; + crc32 = (crc32 >> 8) ^ s_crc_table[(crc32 ^ pByte_buf[3]) & 0xFF]; + pByte_buf += 4; + buf_len -= 4; + } + + while (buf_len) + { + crc32 = (crc32 >> 8) ^ s_crc_table[(crc32 ^ pByte_buf[0]) & 0xFF]; + ++pByte_buf; + --buf_len; + } + + return ~crc32; +} +#endif + + void mz_free(void *p) + { + MZ_FREE(p); + } + + MINIZ_EXPORT void *miniz_def_alloc_func(void *opaque, size_t items, size_t size) + { + (void)opaque, (void)items, (void)size; + return MZ_MALLOC(items * size); + } + MINIZ_EXPORT void miniz_def_free_func(void *opaque, void *address) + { + (void)opaque, (void)address; + MZ_FREE(address); + } + MINIZ_EXPORT void *miniz_def_realloc_func(void *opaque, void *address, size_t items, size_t size) + { + (void)opaque, (void)address, (void)items, (void)size; + return MZ_REALLOC(address, items * size); + } + + const char *mz_version(void) + { + return MZ_VERSION; + } + +#ifndef MINIZ_NO_ZLIB_APIS + +#ifndef MINIZ_NO_DEFLATE_APIS + + int mz_deflateInit(mz_streamp pStream, int level) + { + return mz_deflateInit2(pStream, level, MZ_DEFLATED, MZ_DEFAULT_WINDOW_BITS, 9, MZ_DEFAULT_STRATEGY); + } + + int mz_deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy) + { + tdefl_compressor *pComp; + mz_uint comp_flags = TDEFL_COMPUTE_ADLER32 | tdefl_create_comp_flags_from_zip_params(level, window_bits, strategy); + + if (!pStream) + return MZ_STREAM_ERROR; + if ((method != MZ_DEFLATED) || ((mem_level < 1) || (mem_level > 9)) || ((window_bits != MZ_DEFAULT_WINDOW_BITS) && (-window_bits != MZ_DEFAULT_WINDOW_BITS))) + return MZ_PARAM_ERROR; + + pStream->data_type = 0; + pStream->adler = MZ_ADLER32_INIT; + pStream->msg = NULL; + pStream->reserved = 0; + pStream->total_in = 0; + pStream->total_out = 0; + if (!pStream->zalloc) + pStream->zalloc = miniz_def_alloc_func; + if (!pStream->zfree) + pStream->zfree = miniz_def_free_func; + + pComp = (tdefl_compressor *)pStream->zalloc(pStream->opaque, 1, sizeof(tdefl_compressor)); + if (!pComp) + return MZ_MEM_ERROR; + + pStream->state = (struct mz_internal_state *)pComp; + + if (tdefl_init(pComp, NULL, NULL, comp_flags) != TDEFL_STATUS_OKAY) + { + mz_deflateEnd(pStream); + return MZ_PARAM_ERROR; + } + + return MZ_OK; + } + + int mz_deflateReset(mz_streamp pStream) + { + if ((!pStream) || (!pStream->state) || (!pStream->zalloc) || (!pStream->zfree)) + return MZ_STREAM_ERROR; + pStream->total_in = pStream->total_out = 0; + tdefl_init((tdefl_compressor *)pStream->state, NULL, NULL, ((tdefl_compressor *)pStream->state)->m_flags); + return MZ_OK; + } + + int mz_deflate(mz_streamp pStream, int flush) + { + size_t in_bytes, out_bytes; + mz_ulong orig_total_in, orig_total_out; + int mz_status = MZ_OK; + + if ((!pStream) || (!pStream->state) || (flush < 0) || (flush > MZ_FINISH) || (!pStream->next_out)) + return MZ_STREAM_ERROR; + if (!pStream->avail_out) + return MZ_BUF_ERROR; + + if (flush == MZ_PARTIAL_FLUSH) + flush = MZ_SYNC_FLUSH; + + if (((tdefl_compressor *)pStream->state)->m_prev_return_status == TDEFL_STATUS_DONE) + return (flush == MZ_FINISH) ? MZ_STREAM_END : MZ_BUF_ERROR; + + orig_total_in = pStream->total_in; + orig_total_out = pStream->total_out; + for (;;) + { + tdefl_status defl_status; + in_bytes = pStream->avail_in; + out_bytes = pStream->avail_out; + + defl_status = tdefl_compress((tdefl_compressor *)pStream->state, pStream->next_in, &in_bytes, pStream->next_out, &out_bytes, (tdefl_flush)flush); + pStream->next_in += (mz_uint)in_bytes; + pStream->avail_in -= (mz_uint)in_bytes; + pStream->total_in += (mz_uint)in_bytes; + pStream->adler = tdefl_get_adler32((tdefl_compressor *)pStream->state); + + pStream->next_out += (mz_uint)out_bytes; + pStream->avail_out -= (mz_uint)out_bytes; + pStream->total_out += (mz_uint)out_bytes; + + if (defl_status < 0) + { + mz_status = MZ_STREAM_ERROR; + break; + } + else if (defl_status == TDEFL_STATUS_DONE) + { + mz_status = MZ_STREAM_END; + break; + } + else if (!pStream->avail_out) + break; + else if ((!pStream->avail_in) && (flush != MZ_FINISH)) + { + if ((flush) || (pStream->total_in != orig_total_in) || (pStream->total_out != orig_total_out)) + break; + return MZ_BUF_ERROR; /* Can't make forward progress without some input. + */ + } + } + return mz_status; + } + + int mz_deflateEnd(mz_streamp pStream) + { + if (!pStream) + return MZ_STREAM_ERROR; + if (pStream->state) + { + pStream->zfree(pStream->opaque, pStream->state); + pStream->state = NULL; + } + return MZ_OK; + } + + mz_ulong mz_deflateBound(mz_streamp pStream, mz_ulong source_len) + { + (void)pStream; + /* This is really over conservative. (And lame, but it's actually pretty tricky to compute a true upper bound given the way tdefl's blocking works.) */ + return MZ_MAX(128 + (source_len * 110) / 100, 128 + source_len + ((source_len / (31 * 1024)) + 1) * 5); + } + + int mz_compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level) + { + int status; + mz_stream stream; + memset(&stream, 0, sizeof(stream)); + + /* In case mz_ulong is 64-bits (argh I hate longs). */ + if ((mz_uint64)(source_len | *pDest_len) > 0xFFFFFFFFU) + return MZ_PARAM_ERROR; + + stream.next_in = pSource; + stream.avail_in = (mz_uint32)source_len; + stream.next_out = pDest; + stream.avail_out = (mz_uint32)*pDest_len; + + status = mz_deflateInit(&stream, level); + if (status != MZ_OK) + return status; + + status = mz_deflate(&stream, MZ_FINISH); + if (status != MZ_STREAM_END) + { + mz_deflateEnd(&stream); + return (status == MZ_OK) ? MZ_BUF_ERROR : status; + } + + *pDest_len = stream.total_out; + return mz_deflateEnd(&stream); + } + + int mz_compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len) + { + return mz_compress2(pDest, pDest_len, pSource, source_len, MZ_DEFAULT_COMPRESSION); + } + + mz_ulong mz_compressBound(mz_ulong source_len) + { + return mz_deflateBound(NULL, source_len); + } + +#endif /*#ifndef MINIZ_NO_DEFLATE_APIS*/ + +#ifndef MINIZ_NO_INFLATE_APIS + + typedef struct + { + tinfl_decompressor m_decomp; + mz_uint m_dict_ofs, m_dict_avail, m_first_call, m_has_flushed; + int m_window_bits; + mz_uint8 m_dict[TINFL_LZ_DICT_SIZE]; + tinfl_status m_last_status; + } inflate_state; + + int mz_inflateInit2(mz_streamp pStream, int window_bits) + { + inflate_state *pDecomp; + if (!pStream) + return MZ_STREAM_ERROR; + if ((window_bits != MZ_DEFAULT_WINDOW_BITS) && (-window_bits != MZ_DEFAULT_WINDOW_BITS)) + return MZ_PARAM_ERROR; + + pStream->data_type = 0; + pStream->adler = 0; + pStream->msg = NULL; + pStream->total_in = 0; + pStream->total_out = 0; + pStream->reserved = 0; + if (!pStream->zalloc) + pStream->zalloc = miniz_def_alloc_func; + if (!pStream->zfree) + pStream->zfree = miniz_def_free_func; + + pDecomp = (inflate_state *)pStream->zalloc(pStream->opaque, 1, sizeof(inflate_state)); + if (!pDecomp) + return MZ_MEM_ERROR; + + pStream->state = (struct mz_internal_state *)pDecomp; + + tinfl_init(&pDecomp->m_decomp); + pDecomp->m_dict_ofs = 0; + pDecomp->m_dict_avail = 0; + pDecomp->m_last_status = TINFL_STATUS_NEEDS_MORE_INPUT; + pDecomp->m_first_call = 1; + pDecomp->m_has_flushed = 0; + pDecomp->m_window_bits = window_bits; + + return MZ_OK; + } + + int mz_inflateInit(mz_streamp pStream) + { + return mz_inflateInit2(pStream, MZ_DEFAULT_WINDOW_BITS); + } + + int mz_inflateReset(mz_streamp pStream) + { + inflate_state *pDecomp; + if (!pStream) + return MZ_STREAM_ERROR; + + pStream->data_type = 0; + pStream->adler = 0; + pStream->msg = NULL; + pStream->total_in = 0; + pStream->total_out = 0; + pStream->reserved = 0; + + pDecomp = (inflate_state *)pStream->state; + + tinfl_init(&pDecomp->m_decomp); + pDecomp->m_dict_ofs = 0; + pDecomp->m_dict_avail = 0; + pDecomp->m_last_status = TINFL_STATUS_NEEDS_MORE_INPUT; + pDecomp->m_first_call = 1; + pDecomp->m_has_flushed = 0; + /* pDecomp->m_window_bits = window_bits */; + + return MZ_OK; + } + + int mz_inflate(mz_streamp pStream, int flush) + { + inflate_state *pState; + mz_uint n, first_call, decomp_flags = TINFL_FLAG_COMPUTE_ADLER32; + size_t in_bytes, out_bytes, orig_avail_in; + tinfl_status status; + + if ((!pStream) || (!pStream->state)) + return MZ_STREAM_ERROR; + if (flush == MZ_PARTIAL_FLUSH) + flush = MZ_SYNC_FLUSH; + if ((flush) && (flush != MZ_SYNC_FLUSH) && (flush != MZ_FINISH)) + return MZ_STREAM_ERROR; + + pState = (inflate_state *)pStream->state; + if (pState->m_window_bits > 0) + decomp_flags |= TINFL_FLAG_PARSE_ZLIB_HEADER; + orig_avail_in = pStream->avail_in; + + first_call = pState->m_first_call; + pState->m_first_call = 0; + if (pState->m_last_status < 0) + return MZ_DATA_ERROR; + + if (pState->m_has_flushed && (flush != MZ_FINISH)) + return MZ_STREAM_ERROR; + pState->m_has_flushed |= (flush == MZ_FINISH); + + if ((flush == MZ_FINISH) && (first_call)) + { + /* MZ_FINISH on the first call implies that the input and output buffers are large enough to hold the entire compressed/decompressed file. */ + decomp_flags |= TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF; + in_bytes = pStream->avail_in; + out_bytes = pStream->avail_out; + status = tinfl_decompress(&pState->m_decomp, pStream->next_in, &in_bytes, pStream->next_out, pStream->next_out, &out_bytes, decomp_flags); + pState->m_last_status = status; + pStream->next_in += (mz_uint)in_bytes; + pStream->avail_in -= (mz_uint)in_bytes; + pStream->total_in += (mz_uint)in_bytes; + pStream->adler = tinfl_get_adler32(&pState->m_decomp); + pStream->next_out += (mz_uint)out_bytes; + pStream->avail_out -= (mz_uint)out_bytes; + pStream->total_out += (mz_uint)out_bytes; + + if (status < 0) + return MZ_DATA_ERROR; + else if (status != TINFL_STATUS_DONE) + { + pState->m_last_status = TINFL_STATUS_FAILED; + return MZ_BUF_ERROR; + } + return MZ_STREAM_END; + } + /* flush != MZ_FINISH then we must assume there's more input. */ + if (flush != MZ_FINISH) + decomp_flags |= TINFL_FLAG_HAS_MORE_INPUT; + + if (pState->m_dict_avail) + { + n = MZ_MIN(pState->m_dict_avail, pStream->avail_out); + memcpy(pStream->next_out, pState->m_dict + pState->m_dict_ofs, n); + pStream->next_out += n; + pStream->avail_out -= n; + pStream->total_out += n; + pState->m_dict_avail -= n; + pState->m_dict_ofs = (pState->m_dict_ofs + n) & (TINFL_LZ_DICT_SIZE - 1); + return ((pState->m_last_status == TINFL_STATUS_DONE) && (!pState->m_dict_avail)) ? MZ_STREAM_END : MZ_OK; + } + + for (;;) + { + in_bytes = pStream->avail_in; + out_bytes = TINFL_LZ_DICT_SIZE - pState->m_dict_ofs; + + status = tinfl_decompress(&pState->m_decomp, pStream->next_in, &in_bytes, pState->m_dict, pState->m_dict + pState->m_dict_ofs, &out_bytes, decomp_flags); + pState->m_last_status = status; + + pStream->next_in += (mz_uint)in_bytes; + pStream->avail_in -= (mz_uint)in_bytes; + pStream->total_in += (mz_uint)in_bytes; + pStream->adler = tinfl_get_adler32(&pState->m_decomp); + + pState->m_dict_avail = (mz_uint)out_bytes; + + n = MZ_MIN(pState->m_dict_avail, pStream->avail_out); + memcpy(pStream->next_out, pState->m_dict + pState->m_dict_ofs, n); + pStream->next_out += n; + pStream->avail_out -= n; + pStream->total_out += n; + pState->m_dict_avail -= n; + pState->m_dict_ofs = (pState->m_dict_ofs + n) & (TINFL_LZ_DICT_SIZE - 1); + + if (status < 0) + return MZ_DATA_ERROR; /* Stream is corrupted (there could be some uncompressed data left in the output dictionary - oh well). */ + else if ((status == TINFL_STATUS_NEEDS_MORE_INPUT) && (!orig_avail_in)) + return MZ_BUF_ERROR; /* Signal caller that we can't make forward progress without supplying more input or by setting flush to MZ_FINISH. */ + else if (flush == MZ_FINISH) + { + /* The output buffer MUST be large to hold the remaining uncompressed data when flush==MZ_FINISH. */ + if (status == TINFL_STATUS_DONE) + return pState->m_dict_avail ? MZ_BUF_ERROR : MZ_STREAM_END; + /* status here must be TINFL_STATUS_HAS_MORE_OUTPUT, which means there's at least 1 more byte on the way. If there's no more room left in the output buffer then something is wrong. */ + else if (!pStream->avail_out) + return MZ_BUF_ERROR; + } + else if ((status == TINFL_STATUS_DONE) || (!pStream->avail_in) || (!pStream->avail_out) || (pState->m_dict_avail)) + break; + } + + return ((status == TINFL_STATUS_DONE) && (!pState->m_dict_avail)) ? MZ_STREAM_END : MZ_OK; + } + + int mz_inflateEnd(mz_streamp pStream) + { + if (!pStream) + return MZ_STREAM_ERROR; + if (pStream->state) + { + pStream->zfree(pStream->opaque, pStream->state); + pStream->state = NULL; + } + return MZ_OK; + } + int mz_uncompress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong *pSource_len) + { + mz_stream stream; + int status; + memset(&stream, 0, sizeof(stream)); + + /* In case mz_ulong is 64-bits (argh I hate longs). */ + if ((mz_uint64)(*pSource_len | *pDest_len) > 0xFFFFFFFFU) + return MZ_PARAM_ERROR; + + stream.next_in = pSource; + stream.avail_in = (mz_uint32)*pSource_len; + stream.next_out = pDest; + stream.avail_out = (mz_uint32)*pDest_len; + + status = mz_inflateInit(&stream); + if (status != MZ_OK) + return status; + + status = mz_inflate(&stream, MZ_FINISH); + *pSource_len = *pSource_len - stream.avail_in; + if (status != MZ_STREAM_END) + { + mz_inflateEnd(&stream); + return ((status == MZ_BUF_ERROR) && (!stream.avail_in)) ? MZ_DATA_ERROR : status; + } + *pDest_len = stream.total_out; + + return mz_inflateEnd(&stream); + } + + int mz_uncompress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len) + { + return mz_uncompress2(pDest, pDest_len, pSource, &source_len); + } + +#endif /*#ifndef MINIZ_NO_INFLATE_APIS*/ + + const char *mz_error(int err) + { + static struct + { + int m_err; + const char *m_pDesc; + } s_error_descs[] = { + { MZ_OK, "" }, { MZ_STREAM_END, "stream end" }, { MZ_NEED_DICT, "need dictionary" }, { MZ_ERRNO, "file error" }, { MZ_STREAM_ERROR, "stream error" }, { MZ_DATA_ERROR, "data error" }, { MZ_MEM_ERROR, "out of memory" }, { MZ_BUF_ERROR, "buf error" }, { MZ_VERSION_ERROR, "version error" }, { MZ_PARAM_ERROR, "parameter error" } + }; + mz_uint i; + for (i = 0; i < sizeof(s_error_descs) / sizeof(s_error_descs[0]); ++i) + if (s_error_descs[i].m_err == err) + return s_error_descs[i].m_pDesc; + return NULL; + } + +#endif /*MINIZ_NO_ZLIB_APIS */ + +#ifdef __cplusplus +} +#endif + +/* + This is free and unencumbered software released into the public domain. + + Anyone is free to copy, modify, publish, use, compile, sell, or + distribute this software, either in source code form or as a compiled + binary, for any purpose, commercial or non-commercial, and by any + means. + + In jurisdictions that recognize copyright laws, the author or authors + of this software dedicate any and all copyright interest in the + software to the public domain. We make this dedication for the benefit + of the public at large and to the detriment of our heirs and + successors. We intend this dedication to be an overt act of + relinquishment in perpetuity of all present and future rights to this + software under copyright law. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR + OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + OTHER DEALINGS IN THE SOFTWARE. + + For more information, please refer to +*/ diff --git a/src/helpers/miniz/miniz.h b/src/helpers/miniz/miniz.h new file mode 100644 index 000000000..49c2861f5 --- /dev/null +++ b/src/helpers/miniz/miniz.h @@ -0,0 +1,605 @@ +/* miniz.c 3.1.0 - public domain deflate/inflate, zlib-subset, ZIP reading/writing/appending, PNG writing + See "unlicense" statement at the end of this file. + Rich Geldreich , last updated Oct. 13, 2013 + Implements RFC 1950: http://www.ietf.org/rfc/rfc1950.txt and RFC 1951: http://www.ietf.org/rfc/rfc1951.txt + + Most API's defined in miniz.c are optional. For example, to disable the archive related functions just define + MINIZ_NO_ARCHIVE_APIS, or to get rid of all stdio usage define MINIZ_NO_STDIO (see the list below for more macros). + + * Low-level Deflate/Inflate implementation notes: + + Compression: Use the "tdefl" API's. The compressor supports raw, static, and dynamic blocks, lazy or + greedy parsing, match length filtering, RLE-only, and Huffman-only streams. It performs and compresses + approximately as well as zlib. + + Decompression: Use the "tinfl" API's. The entire decompressor is implemented as a single function + coroutine: see tinfl_decompress(). It supports decompression into a 32KB (or larger power of 2) wrapping buffer, or into a memory + block large enough to hold the entire file. + + The low-level tdefl/tinfl API's do not make any use of dynamic memory allocation. + + * zlib-style API notes: + + miniz.c implements a fairly large subset of zlib. There's enough functionality present for it to be a drop-in + zlib replacement in many apps: + The z_stream struct, optional memory allocation callbacks + deflateInit/deflateInit2/deflate/deflateReset/deflateEnd/deflateBound + inflateInit/inflateInit2/inflate/inflateReset/inflateEnd + compress, compress2, compressBound, uncompress + CRC-32, Adler-32 - Using modern, minimal code size, CPU cache friendly routines. + Supports raw deflate streams or standard zlib streams with adler-32 checking. + + Limitations: + The callback API's are not implemented yet. No support for gzip headers or zlib static dictionaries. + I've tried to closely emulate zlib's various flavors of stream flushing and return status codes, but + there are no guarantees that miniz.c pulls this off perfectly. + + * PNG writing: See the tdefl_write_image_to_png_file_in_memory() function, originally written by + Alex Evans. Supports 1-4 bytes/pixel images. + + * ZIP archive API notes: + + The ZIP archive API's where designed with simplicity and efficiency in mind, with just enough abstraction to + get the job done with minimal fuss. There are simple API's to retrieve file information, read files from + existing archives, create new archives, append new files to existing archives, or clone archive data from + one archive to another. It supports archives located in memory or the heap, on disk (using stdio.h), + or you can specify custom file read/write callbacks. + + - Archive reading: Just call this function to read a single file from a disk archive: + + void *mz_zip_extract_archive_file_to_heap(const char *pZip_filename, const char *pArchive_name, + size_t *pSize, mz_uint zip_flags); + + For more complex cases, use the "mz_zip_reader" functions. Upon opening an archive, the entire central + directory is located and read as-is into memory, and subsequent file access only occurs when reading individual files. + + - Archives file scanning: The simple way is to use this function to scan a loaded archive for a specific file: + + int mz_zip_reader_locate_file(mz_zip_archive *pZip, const char *pName, const char *pComment, mz_uint flags); + + The locate operation can optionally check file comments too, which (as one example) can be used to identify + multiple versions of the same file in an archive. This function uses a simple linear search through the central + directory, so it's not very fast. + + Alternately, you can iterate through all the files in an archive (using mz_zip_reader_get_num_files()) and + retrieve detailed info on each file by calling mz_zip_reader_file_stat(). + + - Archive creation: Use the "mz_zip_writer" functions. The ZIP writer immediately writes compressed file data + to disk and builds an exact image of the central directory in memory. The central directory image is written + all at once at the end of the archive file when the archive is finalized. + + The archive writer can optionally align each file's local header and file data to any power of 2 alignment, + which can be useful when the archive will be read from optical media. Also, the writer supports placing + arbitrary data blobs at the very beginning of ZIP archives. Archives written using either feature are still + readable by any ZIP tool. + + - Archive appending: The simple way to add a single file to an archive is to call this function: + + mz_bool mz_zip_add_mem_to_archive_file_in_place(const char *pZip_filename, const char *pArchive_name, + const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags); + + The archive will be created if it doesn't already exist, otherwise it'll be appended to. + Note the appending is done in-place and is not an atomic operation, so if something goes wrong + during the operation it's possible the archive could be left without a central directory (although the local + file headers and file data will be fine, so the archive will be recoverable). + + For more complex archive modification scenarios: + 1. The safest way is to use a mz_zip_reader to read the existing archive, cloning only those bits you want to + preserve into a new archive using using the mz_zip_writer_add_from_zip_reader() function (which compiles the + compressed file data as-is). When you're done, delete the old archive and rename the newly written archive, and + you're done. This is safe but requires a bunch of temporary disk space or heap memory. + + 2. Or, you can convert an mz_zip_reader in-place to an mz_zip_writer using mz_zip_writer_init_from_reader(), + append new files as needed, then finalize the archive which will write an updated central directory to the + original archive. (This is basically what mz_zip_add_mem_to_archive_file_in_place() does.) There's a + possibility that the archive's central directory could be lost with this method if anything goes wrong, though. + + - ZIP archive support limitations: + No spanning support. Extraction functions can only handle unencrypted, stored or deflated files. + Requires streams capable of seeking. + + * This is a header file library, like stb_image.c. To get only a header file, either cut and paste the + below header, or create miniz.h, #define MINIZ_HEADER_FILE_ONLY, and then include miniz.c from it. + + * Important: For best perf. be sure to customize the below macros for your target platform: + #define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 1 + #define MINIZ_LITTLE_ENDIAN 1 + #define MINIZ_HAS_64BIT_REGISTERS 1 + + * On platforms using glibc, Be sure to "#define _LARGEFILE64_SOURCE 1" before including miniz.c to ensure miniz + uses the 64-bit variants: fopen64(), stat64(), etc. Otherwise you won't be able to process large files + (i.e. 32-bit stat() fails for me on files > 0x7FFFFFFF bytes). +*/ +#pragma once + +#include "miniz_export.h" + +/* Defines to completely disable specific portions of miniz.c: + If all macros here are defined the only functionality remaining will be CRC-32 and adler-32. */ + +/* Define MINIZ_NO_STDIO to disable all usage and any functions which rely on stdio for file I/O. */ +/*#define MINIZ_NO_STDIO */ + +/* If MINIZ_NO_TIME is specified then the ZIP archive functions will not be able to get the current time, or */ +/* get/set file times, and the C run-time funcs that get/set times won't be called. */ +/* The current downside is the times written to your archives will be from 1979. */ +/*#define MINIZ_NO_TIME */ + +/* Define MINIZ_NO_DEFLATE_APIS to disable all compression API's. */ +/*#define MINIZ_NO_DEFLATE_APIS */ + +/* Define MINIZ_NO_INFLATE_APIS to disable all decompression API's. */ +/*#define MINIZ_NO_INFLATE_APIS */ + +/* Define MINIZ_NO_ARCHIVE_APIS to disable all ZIP archive API's. */ +/*#define MINIZ_NO_ARCHIVE_APIS */ + +/* Define MINIZ_NO_ARCHIVE_WRITING_APIS to disable all writing related ZIP archive API's. */ +/*#define MINIZ_NO_ARCHIVE_WRITING_APIS */ + +/* Define MINIZ_NO_ZLIB_APIS to remove all ZLIB-style compression/decompression API's. */ +/*#define MINIZ_NO_ZLIB_APIS */ + +/* Define MINIZ_NO_ZLIB_COMPATIBLE_NAME to disable zlib names, to prevent conflicts against stock zlib. */ +/*#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES */ + +/* Define MINIZ_NO_MALLOC to disable all calls to malloc, free, and realloc. + Note if MINIZ_NO_MALLOC is defined then the user must always provide custom user alloc/free/realloc + callbacks to the zlib and archive API's, and a few stand-alone helper API's which don't provide custom user + functions (such as tdefl_compress_mem_to_heap() and tinfl_decompress_mem_to_heap()) won't work. */ +/*#define MINIZ_NO_MALLOC */ + +#ifdef MINIZ_NO_INFLATE_APIS +#define MINIZ_NO_ARCHIVE_APIS +#endif + +#ifdef MINIZ_NO_DEFLATE_APIS +#define MINIZ_NO_ARCHIVE_WRITING_APIS +#endif + +#if defined(__TINYC__) && (defined(__linux) || defined(__linux__)) +/* TODO: Work around "error: include file 'sys\utime.h' when compiling with tcc on Linux */ +#define MINIZ_NO_TIME +#endif + +#include + +#if !defined(MINIZ_NO_TIME) && !defined(MINIZ_NO_ARCHIVE_APIS) +#include +#endif + +#if defined(_M_IX86) || defined(_M_X64) || defined(__i386__) || defined(__i386) || defined(__i486__) || defined(__i486) || defined(i386) || defined(__ia64__) || defined(__x86_64__) +/* MINIZ_X86_OR_X64_CPU is only used to help set the below macros. */ +#define MINIZ_X86_OR_X64_CPU 1 +#else +#define MINIZ_X86_OR_X64_CPU 0 +#endif + +/* Set MINIZ_LITTLE_ENDIAN only if not set */ +#if !defined(MINIZ_LITTLE_ENDIAN) +#if defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) + +#if (__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__) +/* Set MINIZ_LITTLE_ENDIAN to 1 if the processor is little endian. */ +#define MINIZ_LITTLE_ENDIAN 1 +#else +#define MINIZ_LITTLE_ENDIAN 0 +#endif + +#else + +#if MINIZ_X86_OR_X64_CPU +#define MINIZ_LITTLE_ENDIAN 1 +#else +#define MINIZ_LITTLE_ENDIAN 0 +#endif + +#endif +#endif + +/* Using unaligned loads and stores causes errors when using UBSan */ +#if defined(__has_feature) +#if __has_feature(undefined_behavior_sanitizer) +#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0 +#endif +#endif + +/* Set MINIZ_USE_UNALIGNED_LOADS_AND_STORES only if not set */ +#if !defined(MINIZ_USE_UNALIGNED_LOADS_AND_STORES) +#if MINIZ_X86_OR_X64_CPU +/* Set MINIZ_USE_UNALIGNED_LOADS_AND_STORES to 1 on CPU's that permit efficient integer loads and stores from unaligned addresses. */ +#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0 +#define MINIZ_UNALIGNED_USE_MEMCPY +#else +#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0 +#endif +#endif + +#if defined(_M_X64) || defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__) || defined(__ia64__) || defined(__x86_64__) +/* Set MINIZ_HAS_64BIT_REGISTERS to 1 if operations on 64-bit integers are reasonably fast (and don't involve compiler generated calls to helper functions). */ +#define MINIZ_HAS_64BIT_REGISTERS 1 +#else +#define MINIZ_HAS_64BIT_REGISTERS 0 +#endif + +#ifdef __cplusplus +extern "C" +{ +#endif + + /* ------------------- zlib-style API Definitions. */ + + /* For more compatibility with zlib, miniz.c uses unsigned long for some parameters/struct members. Beware: mz_ulong can be either 32 or 64-bits! */ + typedef unsigned long mz_ulong; + + /* mz_free() internally uses the MZ_FREE() macro (which by default calls free() unless you've modified the MZ_MALLOC macro) to release a block allocated from the heap. */ + MINIZ_EXPORT void mz_free(void *p); + +#define MZ_ADLER32_INIT (1) + /* mz_adler32() returns the initial adler-32 value to use when called with ptr==NULL. */ + MINIZ_EXPORT mz_ulong mz_adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len); + +#define MZ_CRC32_INIT (0) + /* mz_crc32() returns the initial CRC-32 value to use when called with ptr==NULL. */ + MINIZ_EXPORT mz_ulong mz_crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len); + + /* Compression strategies. */ + enum + { + MZ_DEFAULT_STRATEGY = 0, + MZ_FILTERED = 1, + MZ_HUFFMAN_ONLY = 2, + MZ_RLE = 3, + MZ_FIXED = 4 + }; + +/* Method */ +#define MZ_DEFLATED 8 + + /* Heap allocation callbacks. + Note that mz_alloc_func parameter types purposely differ from zlib's: items/size is size_t, not unsigned long. */ + typedef void *(*mz_alloc_func)(void *opaque, size_t items, size_t size); + typedef void (*mz_free_func)(void *opaque, void *address); + typedef void *(*mz_realloc_func)(void *opaque, void *address, size_t items, size_t size); + + /* Compression levels: 0-9 are the standard zlib-style levels, 10 is best possible compression (not zlib compatible, and may be very slow), MZ_DEFAULT_COMPRESSION=MZ_DEFAULT_LEVEL. */ + enum + { + MZ_NO_COMPRESSION = 0, + MZ_BEST_SPEED = 1, + MZ_BEST_COMPRESSION = 9, + MZ_UBER_COMPRESSION = 10, + MZ_DEFAULT_LEVEL = 6, + MZ_DEFAULT_COMPRESSION = -1 + }; + +#define MZ_VERSION "11.3.0" +#define MZ_VERNUM 0xB300 +#define MZ_VER_MAJOR 11 +#define MZ_VER_MINOR 3 +#define MZ_VER_REVISION 0 +#define MZ_VER_SUBREVISION 0 + +#ifndef MINIZ_NO_ZLIB_APIS + + /* Flush values. For typical usage you only need MZ_NO_FLUSH and MZ_FINISH. The other values are for advanced use (refer to the zlib docs). */ + enum + { + MZ_NO_FLUSH = 0, + MZ_PARTIAL_FLUSH = 1, + MZ_SYNC_FLUSH = 2, + MZ_FULL_FLUSH = 3, + MZ_FINISH = 4, + MZ_BLOCK = 5 + }; + + /* Return status codes. MZ_PARAM_ERROR is non-standard. */ + enum + { + MZ_OK = 0, + MZ_STREAM_END = 1, + MZ_NEED_DICT = 2, + MZ_ERRNO = -1, + MZ_STREAM_ERROR = -2, + MZ_DATA_ERROR = -3, + MZ_MEM_ERROR = -4, + MZ_BUF_ERROR = -5, + MZ_VERSION_ERROR = -6, + MZ_PARAM_ERROR = -10000 + }; + +/* Window bits */ +#define MZ_DEFAULT_WINDOW_BITS 15 + + struct mz_internal_state; + + /* Compression/decompression stream struct. */ + typedef struct mz_stream_s + { + const unsigned char *next_in; /* pointer to next byte to read */ + unsigned int avail_in; /* number of bytes available at next_in */ + mz_ulong total_in; /* total number of bytes consumed so far */ + + unsigned char *next_out; /* pointer to next byte to write */ + unsigned int avail_out; /* number of bytes that can be written to next_out */ + mz_ulong total_out; /* total number of bytes produced so far */ + + char *msg; /* error msg (unused) */ + struct mz_internal_state *state; /* internal state, allocated by zalloc/zfree */ + + mz_alloc_func zalloc; /* optional heap allocation function (defaults to malloc) */ + mz_free_func zfree; /* optional heap free function (defaults to free) */ + void *opaque; /* heap alloc function user pointer */ + + int data_type; /* data_type (unused) */ + mz_ulong adler; /* adler32 of the source or uncompressed data */ + mz_ulong reserved; /* not used */ + } mz_stream; + + typedef mz_stream *mz_streamp; + + /* Returns the version string of miniz.c. */ + MINIZ_EXPORT const char *mz_version(void); + +#ifndef MINIZ_NO_DEFLATE_APIS + + /* mz_deflateInit() initializes a compressor with default options: */ + /* Parameters: */ + /* pStream must point to an initialized mz_stream struct. */ + /* level must be between [MZ_NO_COMPRESSION, MZ_BEST_COMPRESSION]. */ + /* level 1 enables a specially optimized compression function that's been optimized purely for performance, not ratio. */ + /* (This special func. is currently only enabled when MINIZ_USE_UNALIGNED_LOADS_AND_STORES and MINIZ_LITTLE_ENDIAN are defined.) */ + /* Return values: */ + /* MZ_OK on success. */ + /* MZ_STREAM_ERROR if the stream is bogus. */ + /* MZ_PARAM_ERROR if the input parameters are bogus. */ + /* MZ_MEM_ERROR on out of memory. */ + MINIZ_EXPORT int mz_deflateInit(mz_streamp pStream, int level); + + /* mz_deflateInit2() is like mz_deflate(), except with more control: */ + /* Additional parameters: */ + /* method must be MZ_DEFLATED */ + /* window_bits must be MZ_DEFAULT_WINDOW_BITS (to wrap the deflate stream with zlib header/adler-32 footer) or -MZ_DEFAULT_WINDOW_BITS (raw deflate/no header or footer) */ + /* mem_level must be between [1, 9] (it's checked but ignored by miniz.c) */ + MINIZ_EXPORT int mz_deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy); + + /* Quickly resets a compressor without having to reallocate anything. Same as calling mz_deflateEnd() followed by mz_deflateInit()/mz_deflateInit2(). */ + MINIZ_EXPORT int mz_deflateReset(mz_streamp pStream); + + /* mz_deflate() compresses the input to output, consuming as much of the input and producing as much output as possible. */ + /* Parameters: */ + /* pStream is the stream to read from and write to. You must initialize/update the next_in, avail_in, next_out, and avail_out members. */ + /* flush may be MZ_NO_FLUSH, MZ_PARTIAL_FLUSH/MZ_SYNC_FLUSH, MZ_FULL_FLUSH, or MZ_FINISH. */ + /* Return values: */ + /* MZ_OK on success (when flushing, or if more input is needed but not available, and/or there's more output to be written but the output buffer is full). */ + /* MZ_STREAM_END if all input has been consumed and all output bytes have been written. Don't call mz_deflate() on the stream anymore. */ + /* MZ_STREAM_ERROR if the stream is bogus. */ + /* MZ_PARAM_ERROR if one of the parameters is invalid. */ + /* MZ_BUF_ERROR if no forward progress is possible because the input and/or output buffers are empty. (Fill up the input buffer or free up some output space and try again.) */ + MINIZ_EXPORT int mz_deflate(mz_streamp pStream, int flush); + + /* mz_deflateEnd() deinitializes a compressor: */ + /* Return values: */ + /* MZ_OK on success. */ + /* MZ_STREAM_ERROR if the stream is bogus. */ + MINIZ_EXPORT int mz_deflateEnd(mz_streamp pStream); + + /* mz_deflateBound() returns a (very) conservative upper bound on the amount of data that could be generated by deflate(), assuming flush is set to only MZ_NO_FLUSH or MZ_FINISH. */ + MINIZ_EXPORT mz_ulong mz_deflateBound(mz_streamp pStream, mz_ulong source_len); + + /* Single-call compression functions mz_compress() and mz_compress2(): */ + /* Returns MZ_OK on success, or one of the error codes from mz_deflate() on failure. */ + MINIZ_EXPORT int mz_compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len); + MINIZ_EXPORT int mz_compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level); + + /* mz_compressBound() returns a (very) conservative upper bound on the amount of data that could be generated by calling mz_compress(). */ + MINIZ_EXPORT mz_ulong mz_compressBound(mz_ulong source_len); + +#endif /*#ifndef MINIZ_NO_DEFLATE_APIS*/ + +#ifndef MINIZ_NO_INFLATE_APIS + + /* Initializes a decompressor. */ + MINIZ_EXPORT int mz_inflateInit(mz_streamp pStream); + + /* mz_inflateInit2() is like mz_inflateInit() with an additional option that controls the window size and whether or not the stream has been wrapped with a zlib header/footer: */ + /* window_bits must be MZ_DEFAULT_WINDOW_BITS (to parse zlib header/footer) or -MZ_DEFAULT_WINDOW_BITS (raw deflate). */ + MINIZ_EXPORT int mz_inflateInit2(mz_streamp pStream, int window_bits); + + /* Quickly resets a compressor without having to reallocate anything. Same as calling mz_inflateEnd() followed by mz_inflateInit()/mz_inflateInit2(). */ + MINIZ_EXPORT int mz_inflateReset(mz_streamp pStream); + + /* Decompresses the input stream to the output, consuming only as much of the input as needed, and writing as much to the output as possible. */ + /* Parameters: */ + /* pStream is the stream to read from and write to. You must initialize/update the next_in, avail_in, next_out, and avail_out members. */ + /* flush may be MZ_NO_FLUSH, MZ_SYNC_FLUSH, or MZ_FINISH. */ + /* On the first call, if flush is MZ_FINISH it's assumed the input and output buffers are both sized large enough to decompress the entire stream in a single call (this is slightly faster). */ + /* MZ_FINISH implies that there are no more source bytes available beside what's already in the input buffer, and that the output buffer is large enough to hold the rest of the decompressed data. */ + /* Return values: */ + /* MZ_OK on success. Either more input is needed but not available, and/or there's more output to be written but the output buffer is full. */ + /* MZ_STREAM_END if all needed input has been consumed and all output bytes have been written. For zlib streams, the adler-32 of the decompressed data has also been verified. */ + /* MZ_STREAM_ERROR if the stream is bogus. */ + /* MZ_DATA_ERROR if the deflate stream is invalid. */ + /* MZ_PARAM_ERROR if one of the parameters is invalid. */ + /* MZ_BUF_ERROR if no forward progress is possible because the input buffer is empty but the inflater needs more input to continue, or if the output buffer is not large enough. Call mz_inflate() again */ + /* with more input data, or with more room in the output buffer (except when using single call decompression, described above). */ + MINIZ_EXPORT int mz_inflate(mz_streamp pStream, int flush); + + /* Deinitializes a decompressor. */ + MINIZ_EXPORT int mz_inflateEnd(mz_streamp pStream); + + /* Single-call decompression. */ + /* Returns MZ_OK on success, or one of the error codes from mz_inflate() on failure. */ + MINIZ_EXPORT int mz_uncompress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len); + MINIZ_EXPORT int mz_uncompress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong *pSource_len); +#endif /*#ifndef MINIZ_NO_INFLATE_APIS*/ + + /* Returns a string description of the specified error code, or NULL if the error code is invalid. */ + MINIZ_EXPORT const char *mz_error(int err); + +/* Redefine zlib-compatible names to miniz equivalents, so miniz.c can be used as a drop-in replacement for the subset of zlib that miniz.c supports. */ +/* Define MINIZ_NO_ZLIB_COMPATIBLE_NAMES to disable zlib-compatibility if you use zlib in the same project. */ +#ifndef MINIZ_NO_ZLIB_COMPATIBLE_NAMES + typedef unsigned char Byte; + typedef unsigned int uInt; + typedef mz_ulong uLong; + typedef Byte Bytef; + typedef uInt uIntf; + typedef char charf; + typedef int intf; + typedef void *voidpf; + typedef uLong uLongf; + typedef void *voidp; + typedef void *const voidpc; +#define Z_NULL 0 +#define Z_NO_FLUSH MZ_NO_FLUSH +#define Z_PARTIAL_FLUSH MZ_PARTIAL_FLUSH +#define Z_SYNC_FLUSH MZ_SYNC_FLUSH +#define Z_FULL_FLUSH MZ_FULL_FLUSH +#define Z_FINISH MZ_FINISH +#define Z_BLOCK MZ_BLOCK +#define Z_OK MZ_OK +#define Z_STREAM_END MZ_STREAM_END +#define Z_NEED_DICT MZ_NEED_DICT +#define Z_ERRNO MZ_ERRNO +#define Z_STREAM_ERROR MZ_STREAM_ERROR +#define Z_DATA_ERROR MZ_DATA_ERROR +#define Z_MEM_ERROR MZ_MEM_ERROR +#define Z_BUF_ERROR MZ_BUF_ERROR +#define Z_VERSION_ERROR MZ_VERSION_ERROR +#define Z_PARAM_ERROR MZ_PARAM_ERROR +#define Z_NO_COMPRESSION MZ_NO_COMPRESSION +#define Z_BEST_SPEED MZ_BEST_SPEED +#define Z_BEST_COMPRESSION MZ_BEST_COMPRESSION +#define Z_DEFAULT_COMPRESSION MZ_DEFAULT_COMPRESSION +#define Z_DEFAULT_STRATEGY MZ_DEFAULT_STRATEGY +#define Z_FILTERED MZ_FILTERED +#define Z_HUFFMAN_ONLY MZ_HUFFMAN_ONLY +#define Z_RLE MZ_RLE +#define Z_FIXED MZ_FIXED +#define Z_DEFLATED MZ_DEFLATED +#define Z_DEFAULT_WINDOW_BITS MZ_DEFAULT_WINDOW_BITS + /* See mz_alloc_func */ + typedef void *(*alloc_func)(void *opaque, size_t items, size_t size); + /* See mz_free_func */ + typedef void (*free_func)(void *opaque, void *address); + +#define internal_state mz_internal_state +#define z_stream mz_stream + +#ifndef MINIZ_NO_DEFLATE_APIS + /* Compatiblity with zlib API. See called functions for documentation */ + static int deflateInit(mz_streamp pStream, int level) + { + return mz_deflateInit(pStream, level); + } + static int deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy) + { + return mz_deflateInit2(pStream, level, method, window_bits, mem_level, strategy); + } + static int deflateReset(mz_streamp pStream) + { + return mz_deflateReset(pStream); + } + static int deflate(mz_streamp pStream, int flush) + { + return mz_deflate(pStream, flush); + } + static int deflateEnd(mz_streamp pStream) + { + return mz_deflateEnd(pStream); + } + static mz_ulong deflateBound(mz_streamp pStream, mz_ulong source_len) + { + return mz_deflateBound(pStream, source_len); + } + static int compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len) + { + return mz_compress(pDest, pDest_len, pSource, source_len); + } + static int compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level) + { + return mz_compress2(pDest, pDest_len, pSource, source_len, level); + } + static mz_ulong compressBound(mz_ulong source_len) + { + return mz_compressBound(source_len); + } +#endif /*#ifndef MINIZ_NO_DEFLATE_APIS*/ + +#ifndef MINIZ_NO_INFLATE_APIS + /* Compatiblity with zlib API. See called functions for documentation */ + static int inflateInit(mz_streamp pStream) + { + return mz_inflateInit(pStream); + } + + static int inflateInit2(mz_streamp pStream, int window_bits) + { + return mz_inflateInit2(pStream, window_bits); + } + + static int inflateReset(mz_streamp pStream) + { + return mz_inflateReset(pStream); + } + + static int inflate(mz_streamp pStream, int flush) + { + return mz_inflate(pStream, flush); + } + + static int inflateEnd(mz_streamp pStream) + { + return mz_inflateEnd(pStream); + } + + static int uncompress(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len) + { + return mz_uncompress(pDest, pDest_len, pSource, source_len); + } + + static int uncompress2(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong* pSource_len) + { + return mz_uncompress2(pDest, pDest_len, pSource, pSource_len); + } +#endif /*#ifndef MINIZ_NO_INFLATE_APIS*/ + + static mz_ulong crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len) + { + return mz_crc32(crc, ptr, buf_len); + } + + static mz_ulong adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len) + { + return mz_adler32(adler, ptr, buf_len); + } + +#define MAX_WBITS 15 +#define MAX_MEM_LEVEL 9 + + static const char* zError(int err) + { + return mz_error(err); + } +#define ZLIB_VERSION MZ_VERSION +#define ZLIB_VERNUM MZ_VERNUM +#define ZLIB_VER_MAJOR MZ_VER_MAJOR +#define ZLIB_VER_MINOR MZ_VER_MINOR +#define ZLIB_VER_REVISION MZ_VER_REVISION +#define ZLIB_VER_SUBREVISION MZ_VER_SUBREVISION + +#define zlibVersion mz_version +#define zlib_version mz_version() +#endif /* #ifndef MINIZ_NO_ZLIB_COMPATIBLE_NAMES */ + +#endif /* MINIZ_NO_ZLIB_APIS */ + +#ifdef __cplusplus +} +#endif + +#include "miniz_common.h" +#include "miniz_tdef.h" +#include "miniz_tinfl.h" +#include "miniz_zip.h" \ No newline at end of file diff --git a/src/helpers/miniz/miniz_common.h b/src/helpers/miniz/miniz_common.h new file mode 100644 index 000000000..05cafaebe --- /dev/null +++ b/src/helpers/miniz/miniz_common.h @@ -0,0 +1,97 @@ +#pragma once +#include +#include +#include +#include + +#include "miniz_export.h" + +/* ------------------- Types and macros */ +typedef unsigned char mz_uint8; +typedef int16_t mz_int16; +typedef uint16_t mz_uint16; +typedef uint32_t mz_uint32; +typedef uint32_t mz_uint; +typedef int64_t mz_int64; +typedef uint64_t mz_uint64; +typedef int mz_bool; + +#define MZ_FALSE (0) +#define MZ_TRUE (1) + +/* Works around MSVC's spammy "warning C4127: conditional expression is constant" message. */ +#ifdef _MSC_VER +#define MZ_MACRO_END while (0, 0) +#else +#define MZ_MACRO_END while (0) +#endif + +#ifdef MINIZ_NO_STDIO +#define MZ_FILE void * +#else +#include +#define MZ_FILE FILE +#endif /* #ifdef MINIZ_NO_STDIO */ + +#ifdef MINIZ_NO_TIME +typedef struct mz_dummy_time_t_tag +{ + mz_uint32 m_dummy1; + mz_uint32 m_dummy2; +} mz_dummy_time_t; +#define MZ_TIME_T mz_dummy_time_t +#else +#define MZ_TIME_T time_t +#endif + +#define MZ_ASSERT(x) assert(x) + +#ifdef MINIZ_NO_MALLOC +#define MZ_MALLOC(x) NULL +#define MZ_FREE(x) (void)x, ((void)0) +#define MZ_REALLOC(p, x) NULL +#else +#define MZ_MALLOC(x) malloc(x) +#define MZ_FREE(x) free(x) +#define MZ_REALLOC(p, x) realloc(p, x) +#endif + +#define MZ_MAX(a, b) (((a) > (b)) ? (a) : (b)) +#define MZ_MIN(a, b) (((a) < (b)) ? (a) : (b)) +#define MZ_CLEAR_OBJ(obj) memset(&(obj), 0, sizeof(obj)) +#define MZ_CLEAR_ARR(obj) memset((obj), 0, sizeof(obj)) +#define MZ_CLEAR_PTR(obj) memset((obj), 0, sizeof(*obj)) + +#if MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN +#define MZ_READ_LE16(p) *((const mz_uint16 *)(p)) +#define MZ_READ_LE32(p) *((const mz_uint32 *)(p)) +#else +#define MZ_READ_LE16(p) ((mz_uint32)(((const mz_uint8 *)(p))[0]) | ((mz_uint32)(((const mz_uint8 *)(p))[1]) << 8U)) +#define MZ_READ_LE32(p) ((mz_uint32)(((const mz_uint8 *)(p))[0]) | ((mz_uint32)(((const mz_uint8 *)(p))[1]) << 8U) | ((mz_uint32)(((const mz_uint8 *)(p))[2]) << 16U) | ((mz_uint32)(((const mz_uint8 *)(p))[3]) << 24U)) +#endif + +#define MZ_READ_LE64(p) (((mz_uint64)MZ_READ_LE32(p)) | (((mz_uint64)MZ_READ_LE32((const mz_uint8 *)(p) + sizeof(mz_uint32))) << 32U)) + +#ifdef _MSC_VER +#define MZ_FORCEINLINE __forceinline +#elif defined(__GNUC__) +#define MZ_FORCEINLINE __inline__ __attribute__((__always_inline__)) +#else +#define MZ_FORCEINLINE inline +#endif + +#ifdef __cplusplus +extern "C" +{ +#endif + + extern MINIZ_EXPORT void *miniz_def_alloc_func(void *opaque, size_t items, size_t size); + extern MINIZ_EXPORT void miniz_def_free_func(void *opaque, void *address); + extern MINIZ_EXPORT void *miniz_def_realloc_func(void *opaque, void *address, size_t items, size_t size); + +#define MZ_UINT16_MAX (0xFFFFU) +#define MZ_UINT32_MAX (0xFFFFFFFFU) + +#ifdef __cplusplus +} +#endif diff --git a/src/helpers/miniz/miniz_export.h b/src/helpers/miniz/miniz_export.h new file mode 100644 index 000000000..e28b6ed21 --- /dev/null +++ b/src/helpers/miniz/miniz_export.h @@ -0,0 +1,14 @@ +#pragma once + +// MINIZ_EXPORT is used for shared library exports +// For embedded/static usage, it's empty +#ifndef MINIZ_EXPORT +#define MINIZ_EXPORT +#endif + +// For NRF52 platforms, detect 32-bit register usage +#if defined(__arm__) || defined(__thumb__) +#define MINIZ_HAS_64BIT_REGISTERS 0 +#else +#define MINIZ_HAS_64BIT_REGISTERS 1 +#endif diff --git a/src/helpers/miniz/miniz_tdef.c b/src/helpers/miniz/miniz_tdef.c new file mode 100644 index 000000000..63506f93c --- /dev/null +++ b/src/helpers/miniz/miniz_tdef.c @@ -0,0 +1,1597 @@ +/************************************************************************** + * + * Copyright 2013-2014 RAD Game Tools and Valve Software + * Copyright 2010-2014 Rich Geldreich and Tenacious Software LLC + * All Rights Reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + **************************************************************************/ + +#include "miniz.h" + +#ifndef MINIZ_NO_DEFLATE_APIS + +#ifdef __cplusplus +extern "C" +{ +#endif + + /* ------------------- Low-level Compression (independent from all decompression API's) */ + + /* Purposely making these tables static for faster init and thread safety. */ + static const mz_uint16 s_tdefl_len_sym[256] = { + 257, 258, 259, 260, 261, 262, 263, 264, 265, 265, 266, 266, 267, 267, 268, 268, 269, 269, 269, 269, 270, 270, 270, 270, 271, 271, 271, 271, 272, 272, 272, 272, + 273, 273, 273, 273, 273, 273, 273, 273, 274, 274, 274, 274, 274, 274, 274, 274, 275, 275, 275, 275, 275, 275, 275, 275, 276, 276, 276, 276, 276, 276, 276, 276, + 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 277, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, 278, + 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 279, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, 280, + 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, 281, + 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, 282, + 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, 283, + 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 284, 285 + }; + + static const mz_uint8 s_tdefl_len_extra[256] = { + 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, + 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, + 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 0 + }; + + static const mz_uint8 s_tdefl_small_dist_sym[512] = { + 0, 1, 2, 3, 4, 4, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, + 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, + 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, + 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, + 14, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, + 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, + 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, + 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, + 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, + 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, + 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, + 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17, 17 + }; + + static const mz_uint8 s_tdefl_small_dist_extra[512] = { + 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, + 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, + 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, + 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, + 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, + 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, + 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, + 7, 7, 7, 7, 7, 7, 7, 7 + }; + + static const mz_uint8 s_tdefl_large_dist_sym[128] = { + 0, 0, 18, 19, 20, 20, 21, 21, 22, 22, 22, 22, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, + 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, 28, + 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29, 29 + }; + + static const mz_uint8 s_tdefl_large_dist_extra[128] = { + 0, 0, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, + 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, + 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13 + }; + + /* Radix sorts tdefl_sym_freq[] array by 16-bit key m_key. Returns ptr to sorted values. */ + typedef struct + { + mz_uint16 m_key, m_sym_index; + } tdefl_sym_freq; + static tdefl_sym_freq *tdefl_radix_sort_syms(mz_uint num_syms, tdefl_sym_freq *pSyms0, tdefl_sym_freq *pSyms1) + { + mz_uint32 total_passes = 2, pass_shift, pass, i, hist[256 * 2]; + tdefl_sym_freq *pCur_syms = pSyms0, *pNew_syms = pSyms1; + MZ_CLEAR_ARR(hist); + for (i = 0; i < num_syms; i++) + { + mz_uint freq = pSyms0[i].m_key; + hist[freq & 0xFF]++; + hist[256 + ((freq >> 8) & 0xFF)]++; + } + while ((total_passes > 1) && (num_syms == hist[(total_passes - 1) * 256])) + total_passes--; + for (pass_shift = 0, pass = 0; pass < total_passes; pass++, pass_shift += 8) + { + const mz_uint32 *pHist = &hist[pass << 8]; + mz_uint offsets[256], cur_ofs = 0; + for (i = 0; i < 256; i++) + { + offsets[i] = cur_ofs; + cur_ofs += pHist[i]; + } + for (i = 0; i < num_syms; i++) + pNew_syms[offsets[(pCur_syms[i].m_key >> pass_shift) & 0xFF]++] = pCur_syms[i]; + { + tdefl_sym_freq *t = pCur_syms; + pCur_syms = pNew_syms; + pNew_syms = t; + } + } + return pCur_syms; + } + + /* tdefl_calculate_minimum_redundancy() originally written by: Alistair Moffat, alistair@cs.mu.oz.au, Jyrki Katajainen, jyrki@diku.dk, November 1996. */ + static void tdefl_calculate_minimum_redundancy(tdefl_sym_freq *A, int n) + { + int root, leaf, next, avbl, used, dpth; + if (n == 0) + return; + else if (n == 1) + { + A[0].m_key = 1; + return; + } + A[0].m_key += A[1].m_key; + root = 0; + leaf = 2; + for (next = 1; next < n - 1; next++) + { + if (leaf >= n || A[root].m_key < A[leaf].m_key) + { + A[next].m_key = A[root].m_key; + A[root++].m_key = (mz_uint16)next; + } + else + A[next].m_key = A[leaf++].m_key; + if (leaf >= n || (root < next && A[root].m_key < A[leaf].m_key)) + { + A[next].m_key = (mz_uint16)(A[next].m_key + A[root].m_key); + A[root++].m_key = (mz_uint16)next; + } + else + A[next].m_key = (mz_uint16)(A[next].m_key + A[leaf++].m_key); + } + A[n - 2].m_key = 0; + for (next = n - 3; next >= 0; next--) + A[next].m_key = A[A[next].m_key].m_key + 1; + avbl = 1; + used = dpth = 0; + root = n - 2; + next = n - 1; + while (avbl > 0) + { + while (root >= 0 && (int)A[root].m_key == dpth) + { + used++; + root--; + } + while (avbl > used) + { + A[next--].m_key = (mz_uint16)(dpth); + avbl--; + } + avbl = 2 * used; + dpth++; + used = 0; + } + } + + /* Limits canonical Huffman code table's max code size. */ + enum + { + TDEFL_MAX_SUPPORTED_HUFF_CODESIZE = 32 + }; + static void tdefl_huffman_enforce_max_code_size(int *pNum_codes, int code_list_len, int max_code_size) + { + int i; + mz_uint32 total = 0; + if (code_list_len <= 1) + return; + for (i = max_code_size + 1; i <= TDEFL_MAX_SUPPORTED_HUFF_CODESIZE; i++) + pNum_codes[max_code_size] += pNum_codes[i]; + for (i = max_code_size; i > 0; i--) + total += (((mz_uint32)pNum_codes[i]) << (max_code_size - i)); + while (total != (1UL << max_code_size)) + { + pNum_codes[max_code_size]--; + for (i = max_code_size - 1; i > 0; i--) + if (pNum_codes[i]) + { + pNum_codes[i]--; + pNum_codes[i + 1] += 2; + break; + } + total--; + } + } + + static void tdefl_optimize_huffman_table(tdefl_compressor *d, int table_num, int table_len, int code_size_limit, int static_table) + { + int i, j, l, num_codes[1 + TDEFL_MAX_SUPPORTED_HUFF_CODESIZE]; + mz_uint next_code[TDEFL_MAX_SUPPORTED_HUFF_CODESIZE + 1]; + MZ_CLEAR_ARR(num_codes); + if (static_table) + { + for (i = 0; i < table_len; i++) + num_codes[d->m_huff_code_sizes[table_num][i]]++; + } + else + { + tdefl_sym_freq syms0[TDEFL_MAX_HUFF_SYMBOLS], syms1[TDEFL_MAX_HUFF_SYMBOLS], *pSyms; + int num_used_syms = 0; + const mz_uint16 *pSym_count = &d->m_huff_count[table_num][0]; + for (i = 0; i < table_len; i++) + if (pSym_count[i]) + { + syms0[num_used_syms].m_key = (mz_uint16)pSym_count[i]; + syms0[num_used_syms++].m_sym_index = (mz_uint16)i; + } + + pSyms = tdefl_radix_sort_syms(num_used_syms, syms0, syms1); + tdefl_calculate_minimum_redundancy(pSyms, num_used_syms); + + for (i = 0; i < num_used_syms; i++) + num_codes[pSyms[i].m_key]++; + + tdefl_huffman_enforce_max_code_size(num_codes, num_used_syms, code_size_limit); + + MZ_CLEAR_ARR(d->m_huff_code_sizes[table_num]); + MZ_CLEAR_ARR(d->m_huff_codes[table_num]); + for (i = 1, j = num_used_syms; i <= code_size_limit; i++) + for (l = num_codes[i]; l > 0; l--) + d->m_huff_code_sizes[table_num][pSyms[--j].m_sym_index] = (mz_uint8)(i); + } + + next_code[1] = 0; + for (j = 0, i = 2; i <= code_size_limit; i++) + next_code[i] = j = ((j + num_codes[i - 1]) << 1); + + for (i = 0; i < table_len; i++) + { + mz_uint rev_code = 0, code, code_size; + if ((code_size = d->m_huff_code_sizes[table_num][i]) == 0) + continue; + code = next_code[code_size]++; + for (l = code_size; l > 0; l--, code >>= 1) + rev_code = (rev_code << 1) | (code & 1); + d->m_huff_codes[table_num][i] = (mz_uint16)rev_code; + } + } + +#define TDEFL_PUT_BITS(b, l) \ + do \ + { \ + mz_uint bits = b; \ + mz_uint len = l; \ + MZ_ASSERT(bits <= ((1U << len) - 1U)); \ + d->m_bit_buffer |= (bits << d->m_bits_in); \ + d->m_bits_in += len; \ + while (d->m_bits_in >= 8) \ + { \ + if (d->m_pOutput_buf < d->m_pOutput_buf_end) \ + *d->m_pOutput_buf++ = (mz_uint8)(d->m_bit_buffer); \ + d->m_bit_buffer >>= 8; \ + d->m_bits_in -= 8; \ + } \ + } \ + MZ_MACRO_END + +#define TDEFL_RLE_PREV_CODE_SIZE() \ + { \ + if (rle_repeat_count) \ + { \ + if (rle_repeat_count < 3) \ + { \ + d->m_huff_count[2][prev_code_size] = (mz_uint16)(d->m_huff_count[2][prev_code_size] + rle_repeat_count); \ + while (rle_repeat_count--) \ + packed_code_sizes[num_packed_code_sizes++] = prev_code_size; \ + } \ + else \ + { \ + d->m_huff_count[2][16] = (mz_uint16)(d->m_huff_count[2][16] + 1); \ + packed_code_sizes[num_packed_code_sizes++] = 16; \ + packed_code_sizes[num_packed_code_sizes++] = (mz_uint8)(rle_repeat_count - 3); \ + } \ + rle_repeat_count = 0; \ + } \ + } + +#define TDEFL_RLE_ZERO_CODE_SIZE() \ + { \ + if (rle_z_count) \ + { \ + if (rle_z_count < 3) \ + { \ + d->m_huff_count[2][0] = (mz_uint16)(d->m_huff_count[2][0] + rle_z_count); \ + while (rle_z_count--) \ + packed_code_sizes[num_packed_code_sizes++] = 0; \ + } \ + else if (rle_z_count <= 10) \ + { \ + d->m_huff_count[2][17] = (mz_uint16)(d->m_huff_count[2][17] + 1); \ + packed_code_sizes[num_packed_code_sizes++] = 17; \ + packed_code_sizes[num_packed_code_sizes++] = (mz_uint8)(rle_z_count - 3); \ + } \ + else \ + { \ + d->m_huff_count[2][18] = (mz_uint16)(d->m_huff_count[2][18] + 1); \ + packed_code_sizes[num_packed_code_sizes++] = 18; \ + packed_code_sizes[num_packed_code_sizes++] = (mz_uint8)(rle_z_count - 11); \ + } \ + rle_z_count = 0; \ + } \ + } + + static const mz_uint8 s_tdefl_packed_code_size_syms_swizzle[] = { 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 }; + + static void tdefl_start_dynamic_block(tdefl_compressor *d) + { + int num_lit_codes, num_dist_codes, num_bit_lengths; + mz_uint i, total_code_sizes_to_pack, num_packed_code_sizes, rle_z_count, rle_repeat_count, packed_code_sizes_index; + mz_uint8 code_sizes_to_pack[TDEFL_MAX_HUFF_SYMBOLS_0 + TDEFL_MAX_HUFF_SYMBOLS_1], packed_code_sizes[TDEFL_MAX_HUFF_SYMBOLS_0 + TDEFL_MAX_HUFF_SYMBOLS_1], prev_code_size = 0xFF; + + d->m_huff_count[0][256] = 1; + + tdefl_optimize_huffman_table(d, 0, TDEFL_MAX_HUFF_SYMBOLS_0, 15, MZ_FALSE); + tdefl_optimize_huffman_table(d, 1, TDEFL_MAX_HUFF_SYMBOLS_1, 15, MZ_FALSE); + + for (num_lit_codes = 286; num_lit_codes > 257; num_lit_codes--) + if (d->m_huff_code_sizes[0][num_lit_codes - 1]) + break; + for (num_dist_codes = 30; num_dist_codes > 1; num_dist_codes--) + if (d->m_huff_code_sizes[1][num_dist_codes - 1]) + break; + + memcpy(code_sizes_to_pack, &d->m_huff_code_sizes[0][0], num_lit_codes); + memcpy(code_sizes_to_pack + num_lit_codes, &d->m_huff_code_sizes[1][0], num_dist_codes); + total_code_sizes_to_pack = num_lit_codes + num_dist_codes; + num_packed_code_sizes = 0; + rle_z_count = 0; + rle_repeat_count = 0; + + memset(&d->m_huff_count[2][0], 0, sizeof(d->m_huff_count[2][0]) * TDEFL_MAX_HUFF_SYMBOLS_2); + for (i = 0; i < total_code_sizes_to_pack; i++) + { + mz_uint8 code_size = code_sizes_to_pack[i]; + if (!code_size) + { + TDEFL_RLE_PREV_CODE_SIZE(); + if (++rle_z_count == 138) + { + TDEFL_RLE_ZERO_CODE_SIZE(); + } + } + else + { + TDEFL_RLE_ZERO_CODE_SIZE(); + if (code_size != prev_code_size) + { + TDEFL_RLE_PREV_CODE_SIZE(); + d->m_huff_count[2][code_size] = (mz_uint16)(d->m_huff_count[2][code_size] + 1); + packed_code_sizes[num_packed_code_sizes++] = code_size; + } + else if (++rle_repeat_count == 6) + { + TDEFL_RLE_PREV_CODE_SIZE(); + } + } + prev_code_size = code_size; + } + if (rle_repeat_count) + { + TDEFL_RLE_PREV_CODE_SIZE(); + } + else + { + TDEFL_RLE_ZERO_CODE_SIZE(); + } + + tdefl_optimize_huffman_table(d, 2, TDEFL_MAX_HUFF_SYMBOLS_2, 7, MZ_FALSE); + + TDEFL_PUT_BITS(2, 2); + + TDEFL_PUT_BITS(num_lit_codes - 257, 5); + TDEFL_PUT_BITS(num_dist_codes - 1, 5); + + for (num_bit_lengths = 18; num_bit_lengths >= 0; num_bit_lengths--) + if (d->m_huff_code_sizes[2][s_tdefl_packed_code_size_syms_swizzle[num_bit_lengths]]) + break; + num_bit_lengths = MZ_MAX(4, (num_bit_lengths + 1)); + TDEFL_PUT_BITS(num_bit_lengths - 4, 4); + for (i = 0; (int)i < num_bit_lengths; i++) + TDEFL_PUT_BITS(d->m_huff_code_sizes[2][s_tdefl_packed_code_size_syms_swizzle[i]], 3); + + for (packed_code_sizes_index = 0; packed_code_sizes_index < num_packed_code_sizes;) + { + mz_uint code = packed_code_sizes[packed_code_sizes_index++]; + MZ_ASSERT(code < TDEFL_MAX_HUFF_SYMBOLS_2); + TDEFL_PUT_BITS(d->m_huff_codes[2][code], d->m_huff_code_sizes[2][code]); + if (code >= 16) + TDEFL_PUT_BITS(packed_code_sizes[packed_code_sizes_index++], "\02\03\07"[code - 16]); + } + } + + static void tdefl_start_static_block(tdefl_compressor *d) + { + mz_uint i; + mz_uint8 *p = &d->m_huff_code_sizes[0][0]; + + for (i = 0; i <= 143; ++i) + *p++ = 8; + for (; i <= 255; ++i) + *p++ = 9; + for (; i <= 279; ++i) + *p++ = 7; + for (; i <= 287; ++i) + *p++ = 8; + + memset(d->m_huff_code_sizes[1], 5, 32); + + tdefl_optimize_huffman_table(d, 0, 288, 15, MZ_TRUE); + tdefl_optimize_huffman_table(d, 1, 32, 15, MZ_TRUE); + + TDEFL_PUT_BITS(1, 2); + } + + static const mz_uint mz_bitmasks[17] = { 0x0000, 0x0001, 0x0003, 0x0007, 0x000F, 0x001F, 0x003F, 0x007F, 0x00FF, 0x01FF, 0x03FF, 0x07FF, 0x0FFF, 0x1FFF, 0x3FFF, 0x7FFF, 0xFFFF }; + +#if MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN && MINIZ_HAS_64BIT_REGISTERS + static mz_bool tdefl_compress_lz_codes(tdefl_compressor *d) + { + mz_uint flags; + mz_uint8 *pLZ_codes; + mz_uint8 *pOutput_buf = d->m_pOutput_buf; + mz_uint8 *pLZ_code_buf_end = d->m_pLZ_code_buf; + mz_uint64 bit_buffer = d->m_bit_buffer; + mz_uint bits_in = d->m_bits_in; + +#define TDEFL_PUT_BITS_FAST(b, l) \ + { \ + bit_buffer |= (((mz_uint64)(b)) << bits_in); \ + bits_in += (l); \ + } + + flags = 1; + for (pLZ_codes = d->m_lz_code_buf; pLZ_codes < pLZ_code_buf_end; flags >>= 1) + { + if (flags == 1) + flags = *pLZ_codes++ | 0x100; + + if (flags & 1) + { + mz_uint s0, s1, n0, n1, sym, num_extra_bits; + mz_uint match_len = pLZ_codes[0]; + mz_uint match_dist = (pLZ_codes[1] | (pLZ_codes[2] << 8)); + pLZ_codes += 3; + + MZ_ASSERT(d->m_huff_code_sizes[0][s_tdefl_len_sym[match_len]]); + TDEFL_PUT_BITS_FAST(d->m_huff_codes[0][s_tdefl_len_sym[match_len]], d->m_huff_code_sizes[0][s_tdefl_len_sym[match_len]]); + TDEFL_PUT_BITS_FAST(match_len & mz_bitmasks[s_tdefl_len_extra[match_len]], s_tdefl_len_extra[match_len]); + + /* This sequence coaxes MSVC into using cmov's vs. jmp's. */ + s0 = s_tdefl_small_dist_sym[match_dist & 511]; + n0 = s_tdefl_small_dist_extra[match_dist & 511]; + s1 = s_tdefl_large_dist_sym[match_dist >> 8]; + n1 = s_tdefl_large_dist_extra[match_dist >> 8]; + sym = (match_dist < 512) ? s0 : s1; + num_extra_bits = (match_dist < 512) ? n0 : n1; + + MZ_ASSERT(d->m_huff_code_sizes[1][sym]); + TDEFL_PUT_BITS_FAST(d->m_huff_codes[1][sym], d->m_huff_code_sizes[1][sym]); + TDEFL_PUT_BITS_FAST(match_dist & mz_bitmasks[num_extra_bits], num_extra_bits); + } + else + { + mz_uint lit = *pLZ_codes++; + MZ_ASSERT(d->m_huff_code_sizes[0][lit]); + TDEFL_PUT_BITS_FAST(d->m_huff_codes[0][lit], d->m_huff_code_sizes[0][lit]); + + if (((flags & 2) == 0) && (pLZ_codes < pLZ_code_buf_end)) + { + flags >>= 1; + lit = *pLZ_codes++; + MZ_ASSERT(d->m_huff_code_sizes[0][lit]); + TDEFL_PUT_BITS_FAST(d->m_huff_codes[0][lit], d->m_huff_code_sizes[0][lit]); + + if (((flags & 2) == 0) && (pLZ_codes < pLZ_code_buf_end)) + { + flags >>= 1; + lit = *pLZ_codes++; + MZ_ASSERT(d->m_huff_code_sizes[0][lit]); + TDEFL_PUT_BITS_FAST(d->m_huff_codes[0][lit], d->m_huff_code_sizes[0][lit]); + } + } + } + + if (pOutput_buf >= d->m_pOutput_buf_end) + return MZ_FALSE; + + memcpy(pOutput_buf, &bit_buffer, sizeof(mz_uint64)); + pOutput_buf += (bits_in >> 3); + bit_buffer >>= (bits_in & ~7); + bits_in &= 7; + } + +#undef TDEFL_PUT_BITS_FAST + + d->m_pOutput_buf = pOutput_buf; + d->m_bits_in = 0; + d->m_bit_buffer = 0; + + while (bits_in) + { + mz_uint32 n = MZ_MIN(bits_in, 16); + TDEFL_PUT_BITS((mz_uint)bit_buffer & mz_bitmasks[n], n); + bit_buffer >>= n; + bits_in -= n; + } + + TDEFL_PUT_BITS(d->m_huff_codes[0][256], d->m_huff_code_sizes[0][256]); + + return (d->m_pOutput_buf < d->m_pOutput_buf_end); + } +#else +static mz_bool tdefl_compress_lz_codes(tdefl_compressor *d) +{ + mz_uint flags; + mz_uint8 *pLZ_codes; + + flags = 1; + for (pLZ_codes = d->m_lz_code_buf; pLZ_codes < d->m_pLZ_code_buf; flags >>= 1) + { + if (flags == 1) + flags = *pLZ_codes++ | 0x100; + if (flags & 1) + { + mz_uint sym, num_extra_bits; + mz_uint match_len = pLZ_codes[0], match_dist = (pLZ_codes[1] | (pLZ_codes[2] << 8)); + pLZ_codes += 3; + + MZ_ASSERT(d->m_huff_code_sizes[0][s_tdefl_len_sym[match_len]]); + TDEFL_PUT_BITS(d->m_huff_codes[0][s_tdefl_len_sym[match_len]], d->m_huff_code_sizes[0][s_tdefl_len_sym[match_len]]); + TDEFL_PUT_BITS(match_len & mz_bitmasks[s_tdefl_len_extra[match_len]], s_tdefl_len_extra[match_len]); + + if (match_dist < 512) + { + sym = s_tdefl_small_dist_sym[match_dist]; + num_extra_bits = s_tdefl_small_dist_extra[match_dist]; + } + else + { + sym = s_tdefl_large_dist_sym[match_dist >> 8]; + num_extra_bits = s_tdefl_large_dist_extra[match_dist >> 8]; + } + MZ_ASSERT(d->m_huff_code_sizes[1][sym]); + TDEFL_PUT_BITS(d->m_huff_codes[1][sym], d->m_huff_code_sizes[1][sym]); + TDEFL_PUT_BITS(match_dist & mz_bitmasks[num_extra_bits], num_extra_bits); + } + else + { + mz_uint lit = *pLZ_codes++; + MZ_ASSERT(d->m_huff_code_sizes[0][lit]); + TDEFL_PUT_BITS(d->m_huff_codes[0][lit], d->m_huff_code_sizes[0][lit]); + } + } + + TDEFL_PUT_BITS(d->m_huff_codes[0][256], d->m_huff_code_sizes[0][256]); + + return (d->m_pOutput_buf < d->m_pOutput_buf_end); +} +#endif /* MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN && MINIZ_HAS_64BIT_REGISTERS */ + + static mz_bool tdefl_compress_block(tdefl_compressor *d, mz_bool static_block) + { + if (static_block) + tdefl_start_static_block(d); + else + tdefl_start_dynamic_block(d); + return tdefl_compress_lz_codes(d); + } + + static const mz_uint s_tdefl_num_probes[11] = { 0, 1, 6, 32, 16, 32, 128, 256, 512, 768, 1500 }; + + static int tdefl_flush_block(tdefl_compressor *d, int flush) + { + mz_uint saved_bit_buf, saved_bits_in; + mz_uint8 *pSaved_output_buf; + mz_bool comp_block_succeeded = MZ_FALSE; + int n, use_raw_block = ((d->m_flags & TDEFL_FORCE_ALL_RAW_BLOCKS) != 0) && (d->m_lookahead_pos - d->m_lz_code_buf_dict_pos) <= d->m_dict_size; + mz_uint8 *pOutput_buf_start = ((d->m_pPut_buf_func == NULL) && ((*d->m_pOut_buf_size - d->m_out_buf_ofs) >= TDEFL_OUT_BUF_SIZE)) ? ((mz_uint8 *)d->m_pOut_buf + d->m_out_buf_ofs) : d->m_output_buf; + + d->m_pOutput_buf = pOutput_buf_start; + d->m_pOutput_buf_end = d->m_pOutput_buf + TDEFL_OUT_BUF_SIZE - 16; + + MZ_ASSERT(!d->m_output_flush_remaining); + d->m_output_flush_ofs = 0; + d->m_output_flush_remaining = 0; + + *d->m_pLZ_flags = (mz_uint8)(*d->m_pLZ_flags >> d->m_num_flags_left); + d->m_pLZ_code_buf -= (d->m_num_flags_left == 8); + + if ((d->m_flags & TDEFL_WRITE_ZLIB_HEADER) && (!d->m_block_index)) + { + const mz_uint8 cmf = 0x78; + mz_uint8 flg, flevel = 3; + mz_uint header, i, mz_un = sizeof(s_tdefl_num_probes) / sizeof(mz_uint); + + /* Determine compression level by reversing the process in tdefl_create_comp_flags_from_zip_params() */ + for (i = 0; i < mz_un; i++) + if (s_tdefl_num_probes[i] == (d->m_flags & 0xFFF)) + break; + + if (i < 2) + flevel = 0; + else if (i < 6) + flevel = 1; + else if (i == 6) + flevel = 2; + + header = cmf << 8 | (flevel << 6); + header += 31 - (header % 31); + flg = header & 0xFF; + + TDEFL_PUT_BITS(cmf, 8); + TDEFL_PUT_BITS(flg, 8); + } + + TDEFL_PUT_BITS(flush == TDEFL_FINISH, 1); + + pSaved_output_buf = d->m_pOutput_buf; + saved_bit_buf = d->m_bit_buffer; + saved_bits_in = d->m_bits_in; + + if (!use_raw_block) + comp_block_succeeded = tdefl_compress_block(d, (d->m_flags & TDEFL_FORCE_ALL_STATIC_BLOCKS) || (d->m_total_lz_bytes < 48)); + + /* If the block gets expanded, forget the current contents of the output buffer and send a raw block instead. */ + if (((use_raw_block) || ((d->m_total_lz_bytes) && ((d->m_pOutput_buf - pSaved_output_buf + 1U) >= d->m_total_lz_bytes))) && + ((d->m_lookahead_pos - d->m_lz_code_buf_dict_pos) <= d->m_dict_size)) + { + mz_uint i; + d->m_pOutput_buf = pSaved_output_buf; + d->m_bit_buffer = saved_bit_buf, d->m_bits_in = saved_bits_in; + TDEFL_PUT_BITS(0, 2); + if (d->m_bits_in) + { + TDEFL_PUT_BITS(0, 8 - d->m_bits_in); + } + for (i = 2; i; --i, d->m_total_lz_bytes ^= 0xFFFF) + { + TDEFL_PUT_BITS(d->m_total_lz_bytes & 0xFFFF, 16); + } + for (i = 0; i < d->m_total_lz_bytes; ++i) + { + TDEFL_PUT_BITS(d->m_dict[(d->m_lz_code_buf_dict_pos + i) & TDEFL_LZ_DICT_SIZE_MASK], 8); + } + } + /* Check for the extremely unlikely (if not impossible) case of the compressed block not fitting into the output buffer when using dynamic codes. */ + else if (!comp_block_succeeded) + { + d->m_pOutput_buf = pSaved_output_buf; + d->m_bit_buffer = saved_bit_buf, d->m_bits_in = saved_bits_in; + tdefl_compress_block(d, MZ_TRUE); + } + + if (flush) + { + if (flush == TDEFL_FINISH) + { + if (d->m_bits_in) + { + TDEFL_PUT_BITS(0, 8 - d->m_bits_in); + } + if (d->m_flags & TDEFL_WRITE_ZLIB_HEADER) + { + mz_uint i, a = d->m_adler32; + for (i = 0; i < 4; i++) + { + TDEFL_PUT_BITS((a >> 24) & 0xFF, 8); + a <<= 8; + } + } + } + else + { + mz_uint i, z = 0; + TDEFL_PUT_BITS(0, 3); + if (d->m_bits_in) + { + TDEFL_PUT_BITS(0, 8 - d->m_bits_in); + } + for (i = 2; i; --i, z ^= 0xFFFF) + { + TDEFL_PUT_BITS(z & 0xFFFF, 16); + } + } + } + + MZ_ASSERT(d->m_pOutput_buf < d->m_pOutput_buf_end); + + memset(&d->m_huff_count[0][0], 0, sizeof(d->m_huff_count[0][0]) * TDEFL_MAX_HUFF_SYMBOLS_0); + memset(&d->m_huff_count[1][0], 0, sizeof(d->m_huff_count[1][0]) * TDEFL_MAX_HUFF_SYMBOLS_1); + + d->m_pLZ_code_buf = d->m_lz_code_buf + 1; + d->m_pLZ_flags = d->m_lz_code_buf; + d->m_num_flags_left = 8; + d->m_lz_code_buf_dict_pos += d->m_total_lz_bytes; + d->m_total_lz_bytes = 0; + d->m_block_index++; + + if ((n = (int)(d->m_pOutput_buf - pOutput_buf_start)) != 0) + { + if (d->m_pPut_buf_func) + { + *d->m_pIn_buf_size = d->m_pSrc - (const mz_uint8 *)d->m_pIn_buf; + if (!(*d->m_pPut_buf_func)(d->m_output_buf, n, d->m_pPut_buf_user)) + return (d->m_prev_return_status = TDEFL_STATUS_PUT_BUF_FAILED); + } + else if (pOutput_buf_start == d->m_output_buf) + { + int bytes_to_copy = (int)MZ_MIN((size_t)n, (size_t)(*d->m_pOut_buf_size - d->m_out_buf_ofs)); + memcpy((mz_uint8 *)d->m_pOut_buf + d->m_out_buf_ofs, d->m_output_buf, bytes_to_copy); + d->m_out_buf_ofs += bytes_to_copy; + if ((n -= bytes_to_copy) != 0) + { + d->m_output_flush_ofs = bytes_to_copy; + d->m_output_flush_remaining = n; + } + } + else + { + d->m_out_buf_ofs += n; + } + } + + return d->m_output_flush_remaining; + } + +#if MINIZ_USE_UNALIGNED_LOADS_AND_STORES +#ifdef MINIZ_UNALIGNED_USE_MEMCPY + static mz_uint16 TDEFL_READ_UNALIGNED_WORD(const mz_uint8 *p) + { + mz_uint16 ret; + memcpy(&ret, p, sizeof(mz_uint16)); + return ret; + } + static mz_uint16 TDEFL_READ_UNALIGNED_WORD2(const mz_uint16 *p) + { + mz_uint16 ret; + memcpy(&ret, p, sizeof(mz_uint16)); + return ret; + } +#else +#define TDEFL_READ_UNALIGNED_WORD(p) *(const mz_uint16 *)(p) +#define TDEFL_READ_UNALIGNED_WORD2(p) *(const mz_uint16 *)(p) +#endif + static MZ_FORCEINLINE void tdefl_find_match(tdefl_compressor *d, mz_uint lookahead_pos, mz_uint max_dist, mz_uint max_match_len, mz_uint *pMatch_dist, mz_uint *pMatch_len) + { + mz_uint dist, pos = lookahead_pos & TDEFL_LZ_DICT_SIZE_MASK, match_len = *pMatch_len, probe_pos = pos, next_probe_pos, probe_len; + mz_uint num_probes_left = d->m_max_probes[match_len >= 32]; + const mz_uint16 *s = (const mz_uint16 *)(d->m_dict + pos), *p, *q; + mz_uint16 c01 = TDEFL_READ_UNALIGNED_WORD(&d->m_dict[pos + match_len - 1]), s01 = TDEFL_READ_UNALIGNED_WORD2(s); + MZ_ASSERT(max_match_len <= TDEFL_MAX_MATCH_LEN); + if (max_match_len <= match_len) + return; + for (;;) + { + for (;;) + { + if (--num_probes_left == 0) + return; +#define TDEFL_PROBE \ + next_probe_pos = d->m_next[probe_pos]; \ + if ((!next_probe_pos) || ((dist = (mz_uint16)(lookahead_pos - next_probe_pos)) > max_dist)) \ + return; \ + probe_pos = next_probe_pos & TDEFL_LZ_DICT_SIZE_MASK; \ + if (TDEFL_READ_UNALIGNED_WORD(&d->m_dict[probe_pos + match_len - 1]) == c01) \ + break; + TDEFL_PROBE; + TDEFL_PROBE; + TDEFL_PROBE; + } + if (!dist) + break; + q = (const mz_uint16 *)(d->m_dict + probe_pos); + if (TDEFL_READ_UNALIGNED_WORD2(q) != s01) + continue; + p = s; + probe_len = 32; + do + { + } while ((TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && (TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && + (TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && (TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && (--probe_len > 0)); + if (!probe_len) + { + *pMatch_dist = dist; + *pMatch_len = MZ_MIN(max_match_len, (mz_uint)TDEFL_MAX_MATCH_LEN); + break; + } + else if ((probe_len = ((mz_uint)(p - s) * 2) + (mz_uint)(*(const mz_uint8 *)p == *(const mz_uint8 *)q)) > match_len) + { + *pMatch_dist = dist; + if ((*pMatch_len = match_len = MZ_MIN(max_match_len, probe_len)) == max_match_len) + break; + c01 = TDEFL_READ_UNALIGNED_WORD(&d->m_dict[pos + match_len - 1]); + } + } + } +#else +static MZ_FORCEINLINE void tdefl_find_match(tdefl_compressor *d, mz_uint lookahead_pos, mz_uint max_dist, mz_uint max_match_len, mz_uint *pMatch_dist, mz_uint *pMatch_len) +{ + mz_uint dist, pos = lookahead_pos & TDEFL_LZ_DICT_SIZE_MASK, match_len = *pMatch_len, probe_pos = pos, next_probe_pos, probe_len; + mz_uint num_probes_left = d->m_max_probes[match_len >= 32]; + const mz_uint8 *s = d->m_dict + pos, *p, *q; + mz_uint8 c0 = d->m_dict[pos + match_len], c1 = d->m_dict[pos + match_len - 1]; + MZ_ASSERT(max_match_len <= TDEFL_MAX_MATCH_LEN); + if (max_match_len <= match_len) + return; + for (;;) + { + for (;;) + { + if (--num_probes_left == 0) + return; +#define TDEFL_PROBE \ + next_probe_pos = d->m_next[probe_pos]; \ + if ((!next_probe_pos) || ((dist = (mz_uint16)(lookahead_pos - next_probe_pos)) > max_dist)) \ + return; \ + probe_pos = next_probe_pos & TDEFL_LZ_DICT_SIZE_MASK; \ + if ((d->m_dict[probe_pos + match_len] == c0) && (d->m_dict[probe_pos + match_len - 1] == c1)) \ + break; + TDEFL_PROBE; + TDEFL_PROBE; + TDEFL_PROBE; + } + if (!dist) + break; + p = s; + q = d->m_dict + probe_pos; + for (probe_len = 0; probe_len < max_match_len; probe_len++) + if (*p++ != *q++) + break; + if (probe_len > match_len) + { + *pMatch_dist = dist; + if ((*pMatch_len = match_len = probe_len) == max_match_len) + return; + c0 = d->m_dict[pos + match_len]; + c1 = d->m_dict[pos + match_len - 1]; + } + } +} +#endif /* #if MINIZ_USE_UNALIGNED_LOADS_AND_STORES */ + +#if MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN +#ifdef MINIZ_UNALIGNED_USE_MEMCPY + static mz_uint32 TDEFL_READ_UNALIGNED_WORD32(const mz_uint8 *p) + { + mz_uint32 ret; + memcpy(&ret, p, sizeof(mz_uint32)); + return ret; + } +#else +#define TDEFL_READ_UNALIGNED_WORD32(p) *(const mz_uint32 *)(p) +#endif + static mz_bool tdefl_compress_fast(tdefl_compressor *d) + { + /* Faster, minimally featured LZRW1-style match+parse loop with better register utilization. Intended for applications where raw throughput is valued more highly than ratio. */ + mz_uint lookahead_pos = d->m_lookahead_pos, lookahead_size = d->m_lookahead_size, dict_size = d->m_dict_size, total_lz_bytes = d->m_total_lz_bytes, num_flags_left = d->m_num_flags_left; + mz_uint8 *pLZ_code_buf = d->m_pLZ_code_buf, *pLZ_flags = d->m_pLZ_flags; + mz_uint cur_pos = lookahead_pos & TDEFL_LZ_DICT_SIZE_MASK; + + while ((d->m_src_buf_left) || ((d->m_flush) && (lookahead_size))) + { + const mz_uint TDEFL_COMP_FAST_LOOKAHEAD_SIZE = 4096; + mz_uint dst_pos = (lookahead_pos + lookahead_size) & TDEFL_LZ_DICT_SIZE_MASK; + mz_uint num_bytes_to_process = (mz_uint)MZ_MIN(d->m_src_buf_left, TDEFL_COMP_FAST_LOOKAHEAD_SIZE - lookahead_size); + d->m_src_buf_left -= num_bytes_to_process; + lookahead_size += num_bytes_to_process; + + while (num_bytes_to_process) + { + mz_uint32 n = MZ_MIN(TDEFL_LZ_DICT_SIZE - dst_pos, num_bytes_to_process); + memcpy(d->m_dict + dst_pos, d->m_pSrc, n); + if (dst_pos < (TDEFL_MAX_MATCH_LEN - 1)) + memcpy(d->m_dict + TDEFL_LZ_DICT_SIZE + dst_pos, d->m_pSrc, MZ_MIN(n, (TDEFL_MAX_MATCH_LEN - 1) - dst_pos)); + d->m_pSrc += n; + dst_pos = (dst_pos + n) & TDEFL_LZ_DICT_SIZE_MASK; + num_bytes_to_process -= n; + } + + dict_size = MZ_MIN(TDEFL_LZ_DICT_SIZE - lookahead_size, dict_size); + if ((!d->m_flush) && (lookahead_size < TDEFL_COMP_FAST_LOOKAHEAD_SIZE)) + break; + + while (lookahead_size >= 4) + { + mz_uint cur_match_dist, cur_match_len = 1; + mz_uint8 *pCur_dict = d->m_dict + cur_pos; + mz_uint first_trigram = TDEFL_READ_UNALIGNED_WORD32(pCur_dict) & 0xFFFFFF; + mz_uint hash = (first_trigram ^ (first_trigram >> (24 - (TDEFL_LZ_HASH_BITS - 8)))) & TDEFL_LEVEL1_HASH_SIZE_MASK; + mz_uint probe_pos = d->m_hash[hash]; + d->m_hash[hash] = (mz_uint16)lookahead_pos; + + if (((cur_match_dist = (mz_uint16)(lookahead_pos - probe_pos)) <= dict_size) && ((TDEFL_READ_UNALIGNED_WORD32(d->m_dict + (probe_pos &= TDEFL_LZ_DICT_SIZE_MASK)) & 0xFFFFFF) == first_trigram)) + { + const mz_uint16 *p = (const mz_uint16 *)pCur_dict; + const mz_uint16 *q = (const mz_uint16 *)(d->m_dict + probe_pos); + mz_uint32 probe_len = 32; + do + { + } while ((TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && (TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && + (TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && (TDEFL_READ_UNALIGNED_WORD2(++p) == TDEFL_READ_UNALIGNED_WORD2(++q)) && (--probe_len > 0)); + cur_match_len = ((mz_uint)(p - (const mz_uint16 *)pCur_dict) * 2) + (mz_uint)(*(const mz_uint8 *)p == *(const mz_uint8 *)q); + if (!probe_len) + cur_match_len = cur_match_dist ? TDEFL_MAX_MATCH_LEN : 0; + + if ((cur_match_len < TDEFL_MIN_MATCH_LEN) || ((cur_match_len == TDEFL_MIN_MATCH_LEN) && (cur_match_dist >= 8U * 1024U))) + { + cur_match_len = 1; + *pLZ_code_buf++ = (mz_uint8)first_trigram; + *pLZ_flags = (mz_uint8)(*pLZ_flags >> 1); + d->m_huff_count[0][(mz_uint8)first_trigram]++; + } + else + { + mz_uint32 s0, s1; + cur_match_len = MZ_MIN(cur_match_len, lookahead_size); + + MZ_ASSERT((cur_match_len >= TDEFL_MIN_MATCH_LEN) && (cur_match_dist >= 1) && (cur_match_dist <= TDEFL_LZ_DICT_SIZE)); + + cur_match_dist--; + + pLZ_code_buf[0] = (mz_uint8)(cur_match_len - TDEFL_MIN_MATCH_LEN); +#ifdef MINIZ_UNALIGNED_USE_MEMCPY + memcpy(&pLZ_code_buf[1], &cur_match_dist, sizeof(cur_match_dist)); +#else + *(mz_uint16 *)(&pLZ_code_buf[1]) = (mz_uint16)cur_match_dist; +#endif + pLZ_code_buf += 3; + *pLZ_flags = (mz_uint8)((*pLZ_flags >> 1) | 0x80); + + s0 = s_tdefl_small_dist_sym[cur_match_dist & 511]; + s1 = s_tdefl_large_dist_sym[cur_match_dist >> 8]; + d->m_huff_count[1][(cur_match_dist < 512) ? s0 : s1]++; + + d->m_huff_count[0][s_tdefl_len_sym[cur_match_len - TDEFL_MIN_MATCH_LEN]]++; + } + } + else + { + *pLZ_code_buf++ = (mz_uint8)first_trigram; + *pLZ_flags = (mz_uint8)(*pLZ_flags >> 1); + d->m_huff_count[0][(mz_uint8)first_trigram]++; + } + + if (--num_flags_left == 0) + { + num_flags_left = 8; + pLZ_flags = pLZ_code_buf++; + } + + total_lz_bytes += cur_match_len; + lookahead_pos += cur_match_len; + dict_size = MZ_MIN(dict_size + cur_match_len, (mz_uint)TDEFL_LZ_DICT_SIZE); + cur_pos = (cur_pos + cur_match_len) & TDEFL_LZ_DICT_SIZE_MASK; + MZ_ASSERT(lookahead_size >= cur_match_len); + lookahead_size -= cur_match_len; + + if (pLZ_code_buf > &d->m_lz_code_buf[TDEFL_LZ_CODE_BUF_SIZE - 8]) + { + int n; + d->m_lookahead_pos = lookahead_pos; + d->m_lookahead_size = lookahead_size; + d->m_dict_size = dict_size; + d->m_total_lz_bytes = total_lz_bytes; + d->m_pLZ_code_buf = pLZ_code_buf; + d->m_pLZ_flags = pLZ_flags; + d->m_num_flags_left = num_flags_left; + if ((n = tdefl_flush_block(d, 0)) != 0) + return (n < 0) ? MZ_FALSE : MZ_TRUE; + total_lz_bytes = d->m_total_lz_bytes; + pLZ_code_buf = d->m_pLZ_code_buf; + pLZ_flags = d->m_pLZ_flags; + num_flags_left = d->m_num_flags_left; + } + } + + while (lookahead_size) + { + mz_uint8 lit = d->m_dict[cur_pos]; + + total_lz_bytes++; + *pLZ_code_buf++ = lit; + *pLZ_flags = (mz_uint8)(*pLZ_flags >> 1); + if (--num_flags_left == 0) + { + num_flags_left = 8; + pLZ_flags = pLZ_code_buf++; + } + + d->m_huff_count[0][lit]++; + + lookahead_pos++; + dict_size = MZ_MIN(dict_size + 1, (mz_uint)TDEFL_LZ_DICT_SIZE); + cur_pos = (cur_pos + 1) & TDEFL_LZ_DICT_SIZE_MASK; + lookahead_size--; + + if (pLZ_code_buf > &d->m_lz_code_buf[TDEFL_LZ_CODE_BUF_SIZE - 8]) + { + int n; + d->m_lookahead_pos = lookahead_pos; + d->m_lookahead_size = lookahead_size; + d->m_dict_size = dict_size; + d->m_total_lz_bytes = total_lz_bytes; + d->m_pLZ_code_buf = pLZ_code_buf; + d->m_pLZ_flags = pLZ_flags; + d->m_num_flags_left = num_flags_left; + if ((n = tdefl_flush_block(d, 0)) != 0) + return (n < 0) ? MZ_FALSE : MZ_TRUE; + total_lz_bytes = d->m_total_lz_bytes; + pLZ_code_buf = d->m_pLZ_code_buf; + pLZ_flags = d->m_pLZ_flags; + num_flags_left = d->m_num_flags_left; + } + } + } + + d->m_lookahead_pos = lookahead_pos; + d->m_lookahead_size = lookahead_size; + d->m_dict_size = dict_size; + d->m_total_lz_bytes = total_lz_bytes; + d->m_pLZ_code_buf = pLZ_code_buf; + d->m_pLZ_flags = pLZ_flags; + d->m_num_flags_left = num_flags_left; + return MZ_TRUE; + } +#endif /* MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN */ + + static MZ_FORCEINLINE void tdefl_record_literal(tdefl_compressor *d, mz_uint8 lit) + { + d->m_total_lz_bytes++; + *d->m_pLZ_code_buf++ = lit; + *d->m_pLZ_flags = (mz_uint8)(*d->m_pLZ_flags >> 1); + if (--d->m_num_flags_left == 0) + { + d->m_num_flags_left = 8; + d->m_pLZ_flags = d->m_pLZ_code_buf++; + } + d->m_huff_count[0][lit]++; + } + + static MZ_FORCEINLINE void tdefl_record_match(tdefl_compressor *d, mz_uint match_len, mz_uint match_dist) + { + mz_uint32 s0, s1; + + MZ_ASSERT((match_len >= TDEFL_MIN_MATCH_LEN) && (match_dist >= 1) && (match_dist <= TDEFL_LZ_DICT_SIZE)); + + d->m_total_lz_bytes += match_len; + + d->m_pLZ_code_buf[0] = (mz_uint8)(match_len - TDEFL_MIN_MATCH_LEN); + + match_dist -= 1; + d->m_pLZ_code_buf[1] = (mz_uint8)(match_dist & 0xFF); + d->m_pLZ_code_buf[2] = (mz_uint8)(match_dist >> 8); + d->m_pLZ_code_buf += 3; + + *d->m_pLZ_flags = (mz_uint8)((*d->m_pLZ_flags >> 1) | 0x80); + if (--d->m_num_flags_left == 0) + { + d->m_num_flags_left = 8; + d->m_pLZ_flags = d->m_pLZ_code_buf++; + } + + s0 = s_tdefl_small_dist_sym[match_dist & 511]; + s1 = s_tdefl_large_dist_sym[(match_dist >> 8) & 127]; + d->m_huff_count[1][(match_dist < 512) ? s0 : s1]++; + d->m_huff_count[0][s_tdefl_len_sym[match_len - TDEFL_MIN_MATCH_LEN]]++; + } + + static mz_bool tdefl_compress_normal(tdefl_compressor *d) + { + const mz_uint8 *pSrc = d->m_pSrc; + size_t src_buf_left = d->m_src_buf_left; + tdefl_flush flush = d->m_flush; + + while ((src_buf_left) || ((flush) && (d->m_lookahead_size))) + { + mz_uint len_to_move, cur_match_dist, cur_match_len, cur_pos; + /* Update dictionary and hash chains. Keeps the lookahead size equal to TDEFL_MAX_MATCH_LEN. */ + if ((d->m_lookahead_size + d->m_dict_size) >= (TDEFL_MIN_MATCH_LEN - 1)) + { + mz_uint dst_pos = (d->m_lookahead_pos + d->m_lookahead_size) & TDEFL_LZ_DICT_SIZE_MASK, ins_pos = d->m_lookahead_pos + d->m_lookahead_size - 2; + mz_uint hash = (d->m_dict[ins_pos & TDEFL_LZ_DICT_SIZE_MASK] << TDEFL_LZ_HASH_SHIFT) ^ d->m_dict[(ins_pos + 1) & TDEFL_LZ_DICT_SIZE_MASK]; + mz_uint num_bytes_to_process = (mz_uint)MZ_MIN(src_buf_left, TDEFL_MAX_MATCH_LEN - d->m_lookahead_size); + const mz_uint8 *pSrc_end = pSrc ? pSrc + num_bytes_to_process : NULL; + src_buf_left -= num_bytes_to_process; + d->m_lookahead_size += num_bytes_to_process; + while (pSrc != pSrc_end) + { + mz_uint8 c = *pSrc++; + d->m_dict[dst_pos] = c; + if (dst_pos < (TDEFL_MAX_MATCH_LEN - 1)) + d->m_dict[TDEFL_LZ_DICT_SIZE + dst_pos] = c; + hash = ((hash << TDEFL_LZ_HASH_SHIFT) ^ c) & (TDEFL_LZ_HASH_SIZE - 1); + d->m_next[ins_pos & TDEFL_LZ_DICT_SIZE_MASK] = d->m_hash[hash]; + d->m_hash[hash] = (mz_uint16)(ins_pos); + dst_pos = (dst_pos + 1) & TDEFL_LZ_DICT_SIZE_MASK; + ins_pos++; + } + } + else + { + while ((src_buf_left) && (d->m_lookahead_size < TDEFL_MAX_MATCH_LEN)) + { + mz_uint8 c = *pSrc++; + mz_uint dst_pos = (d->m_lookahead_pos + d->m_lookahead_size) & TDEFL_LZ_DICT_SIZE_MASK; + src_buf_left--; + d->m_dict[dst_pos] = c; + if (dst_pos < (TDEFL_MAX_MATCH_LEN - 1)) + d->m_dict[TDEFL_LZ_DICT_SIZE + dst_pos] = c; + if ((++d->m_lookahead_size + d->m_dict_size) >= TDEFL_MIN_MATCH_LEN) + { + mz_uint ins_pos = d->m_lookahead_pos + (d->m_lookahead_size - 1) - 2; + mz_uint hash = ((d->m_dict[ins_pos & TDEFL_LZ_DICT_SIZE_MASK] << (TDEFL_LZ_HASH_SHIFT * 2)) ^ (d->m_dict[(ins_pos + 1) & TDEFL_LZ_DICT_SIZE_MASK] << TDEFL_LZ_HASH_SHIFT) ^ c) & (TDEFL_LZ_HASH_SIZE - 1); + d->m_next[ins_pos & TDEFL_LZ_DICT_SIZE_MASK] = d->m_hash[hash]; + d->m_hash[hash] = (mz_uint16)(ins_pos); + } + } + } + d->m_dict_size = MZ_MIN(TDEFL_LZ_DICT_SIZE - d->m_lookahead_size, d->m_dict_size); + if ((!flush) && (d->m_lookahead_size < TDEFL_MAX_MATCH_LEN)) + break; + + /* Simple lazy/greedy parsing state machine. */ + len_to_move = 1; + cur_match_dist = 0; + cur_match_len = d->m_saved_match_len ? d->m_saved_match_len : (TDEFL_MIN_MATCH_LEN - 1); + cur_pos = d->m_lookahead_pos & TDEFL_LZ_DICT_SIZE_MASK; + if (d->m_flags & (TDEFL_RLE_MATCHES | TDEFL_FORCE_ALL_RAW_BLOCKS)) + { + if ((d->m_dict_size) && (!(d->m_flags & TDEFL_FORCE_ALL_RAW_BLOCKS))) + { + mz_uint8 c = d->m_dict[(cur_pos - 1) & TDEFL_LZ_DICT_SIZE_MASK]; + cur_match_len = 0; + while (cur_match_len < d->m_lookahead_size) + { + if (d->m_dict[cur_pos + cur_match_len] != c) + break; + cur_match_len++; + } + if (cur_match_len < TDEFL_MIN_MATCH_LEN) + cur_match_len = 0; + else + cur_match_dist = 1; + } + } + else + { + tdefl_find_match(d, d->m_lookahead_pos, d->m_dict_size, d->m_lookahead_size, &cur_match_dist, &cur_match_len); + } + if (((cur_match_len == TDEFL_MIN_MATCH_LEN) && (cur_match_dist >= 8U * 1024U)) || (cur_pos == cur_match_dist) || ((d->m_flags & TDEFL_FILTER_MATCHES) && (cur_match_len <= 5))) + { + cur_match_dist = cur_match_len = 0; + } + if (d->m_saved_match_len) + { + if (cur_match_len > d->m_saved_match_len) + { + tdefl_record_literal(d, (mz_uint8)d->m_saved_lit); + if (cur_match_len >= 128) + { + tdefl_record_match(d, cur_match_len, cur_match_dist); + d->m_saved_match_len = 0; + len_to_move = cur_match_len; + } + else + { + d->m_saved_lit = d->m_dict[cur_pos]; + d->m_saved_match_dist = cur_match_dist; + d->m_saved_match_len = cur_match_len; + } + } + else + { + tdefl_record_match(d, d->m_saved_match_len, d->m_saved_match_dist); + len_to_move = d->m_saved_match_len - 1; + d->m_saved_match_len = 0; + } + } + else if (!cur_match_dist) + tdefl_record_literal(d, d->m_dict[MZ_MIN(cur_pos, sizeof(d->m_dict) - 1)]); + else if ((d->m_greedy_parsing) || (d->m_flags & TDEFL_RLE_MATCHES) || (cur_match_len >= 128)) + { + tdefl_record_match(d, cur_match_len, cur_match_dist); + len_to_move = cur_match_len; + } + else + { + d->m_saved_lit = d->m_dict[MZ_MIN(cur_pos, sizeof(d->m_dict) - 1)]; + d->m_saved_match_dist = cur_match_dist; + d->m_saved_match_len = cur_match_len; + } + /* Move the lookahead forward by len_to_move bytes. */ + d->m_lookahead_pos += len_to_move; + MZ_ASSERT(d->m_lookahead_size >= len_to_move); + d->m_lookahead_size -= len_to_move; + d->m_dict_size = MZ_MIN(d->m_dict_size + len_to_move, (mz_uint)TDEFL_LZ_DICT_SIZE); + /* Check if it's time to flush the current LZ codes to the internal output buffer. */ + if ((d->m_pLZ_code_buf > &d->m_lz_code_buf[TDEFL_LZ_CODE_BUF_SIZE - 8]) || + ((d->m_total_lz_bytes > 31 * 1024) && (((((mz_uint)(d->m_pLZ_code_buf - d->m_lz_code_buf) * 115) >> 7) >= d->m_total_lz_bytes) || (d->m_flags & TDEFL_FORCE_ALL_RAW_BLOCKS)))) + { + int n; + d->m_pSrc = pSrc; + d->m_src_buf_left = src_buf_left; + if ((n = tdefl_flush_block(d, 0)) != 0) + return (n < 0) ? MZ_FALSE : MZ_TRUE; + } + } + + d->m_pSrc = pSrc; + d->m_src_buf_left = src_buf_left; + return MZ_TRUE; + } + + static tdefl_status tdefl_flush_output_buffer(tdefl_compressor *d) + { + if (d->m_pIn_buf_size) + { + *d->m_pIn_buf_size = d->m_pSrc - (const mz_uint8 *)d->m_pIn_buf; + } + + if (d->m_pOut_buf_size) + { + size_t n = MZ_MIN(*d->m_pOut_buf_size - d->m_out_buf_ofs, d->m_output_flush_remaining); + memcpy((mz_uint8 *)d->m_pOut_buf + d->m_out_buf_ofs, d->m_output_buf + d->m_output_flush_ofs, n); + d->m_output_flush_ofs += (mz_uint)n; + d->m_output_flush_remaining -= (mz_uint)n; + d->m_out_buf_ofs += n; + + *d->m_pOut_buf_size = d->m_out_buf_ofs; + } + + return (d->m_finished && !d->m_output_flush_remaining) ? TDEFL_STATUS_DONE : TDEFL_STATUS_OKAY; + } + + tdefl_status tdefl_compress(tdefl_compressor *d, const void *pIn_buf, size_t *pIn_buf_size, void *pOut_buf, size_t *pOut_buf_size, tdefl_flush flush) + { + if (!d) + { + if (pIn_buf_size) + *pIn_buf_size = 0; + if (pOut_buf_size) + *pOut_buf_size = 0; + return TDEFL_STATUS_BAD_PARAM; + } + + d->m_pIn_buf = pIn_buf; + d->m_pIn_buf_size = pIn_buf_size; + d->m_pOut_buf = pOut_buf; + d->m_pOut_buf_size = pOut_buf_size; + d->m_pSrc = (const mz_uint8 *)(pIn_buf); + d->m_src_buf_left = pIn_buf_size ? *pIn_buf_size : 0; + d->m_out_buf_ofs = 0; + d->m_flush = flush; + + if (((d->m_pPut_buf_func != NULL) == ((pOut_buf != NULL) || (pOut_buf_size != NULL))) || (d->m_prev_return_status != TDEFL_STATUS_OKAY) || + (d->m_wants_to_finish && (flush != TDEFL_FINISH)) || (pIn_buf_size && *pIn_buf_size && !pIn_buf) || (pOut_buf_size && *pOut_buf_size && !pOut_buf)) + { + if (pIn_buf_size) + *pIn_buf_size = 0; + if (pOut_buf_size) + *pOut_buf_size = 0; + return (d->m_prev_return_status = TDEFL_STATUS_BAD_PARAM); + } + d->m_wants_to_finish |= (flush == TDEFL_FINISH); + + if ((d->m_output_flush_remaining) || (d->m_finished)) + return (d->m_prev_return_status = tdefl_flush_output_buffer(d)); + +#if MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN + if (((d->m_flags & TDEFL_MAX_PROBES_MASK) == 1) && + ((d->m_flags & TDEFL_GREEDY_PARSING_FLAG) != 0) && + ((d->m_flags & (TDEFL_FILTER_MATCHES | TDEFL_FORCE_ALL_RAW_BLOCKS | TDEFL_RLE_MATCHES)) == 0)) + { + if (!tdefl_compress_fast(d)) + return d->m_prev_return_status; + } + else +#endif /* #if MINIZ_USE_UNALIGNED_LOADS_AND_STORES && MINIZ_LITTLE_ENDIAN */ + { + if (!tdefl_compress_normal(d)) + return d->m_prev_return_status; + } + + if ((d->m_flags & (TDEFL_WRITE_ZLIB_HEADER | TDEFL_COMPUTE_ADLER32)) && (pIn_buf)) + d->m_adler32 = (mz_uint32)mz_adler32(d->m_adler32, (const mz_uint8 *)pIn_buf, d->m_pSrc - (const mz_uint8 *)pIn_buf); + + if ((flush) && (!d->m_lookahead_size) && (!d->m_src_buf_left) && (!d->m_output_flush_remaining)) + { + if (tdefl_flush_block(d, flush) < 0) + return d->m_prev_return_status; + d->m_finished = (flush == TDEFL_FINISH); + if (flush == TDEFL_FULL_FLUSH) + { + MZ_CLEAR_ARR(d->m_hash); + MZ_CLEAR_ARR(d->m_next); + d->m_dict_size = 0; + } + } + + return (d->m_prev_return_status = tdefl_flush_output_buffer(d)); + } + + tdefl_status tdefl_compress_buffer(tdefl_compressor *d, const void *pIn_buf, size_t in_buf_size, tdefl_flush flush) + { + MZ_ASSERT(d->m_pPut_buf_func); + return tdefl_compress(d, pIn_buf, &in_buf_size, NULL, NULL, flush); + } + + tdefl_status tdefl_init(tdefl_compressor *d, tdefl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags) + { + d->m_pPut_buf_func = pPut_buf_func; + d->m_pPut_buf_user = pPut_buf_user; + d->m_flags = (mz_uint)(flags); + d->m_max_probes[0] = 1 + ((flags & 0xFFF) + 2) / 3; + d->m_greedy_parsing = (flags & TDEFL_GREEDY_PARSING_FLAG) != 0; + d->m_max_probes[1] = 1 + (((flags & 0xFFF) >> 2) + 2) / 3; + if (!(flags & TDEFL_NONDETERMINISTIC_PARSING_FLAG)) + MZ_CLEAR_ARR(d->m_hash); + d->m_lookahead_pos = d->m_lookahead_size = d->m_dict_size = d->m_total_lz_bytes = d->m_lz_code_buf_dict_pos = d->m_bits_in = 0; + d->m_output_flush_ofs = d->m_output_flush_remaining = d->m_finished = d->m_block_index = d->m_bit_buffer = d->m_wants_to_finish = 0; + d->m_pLZ_code_buf = d->m_lz_code_buf + 1; + d->m_pLZ_flags = d->m_lz_code_buf; + *d->m_pLZ_flags = 0; + d->m_num_flags_left = 8; + d->m_pOutput_buf = d->m_output_buf; + d->m_pOutput_buf_end = d->m_output_buf; + d->m_prev_return_status = TDEFL_STATUS_OKAY; + d->m_saved_match_dist = d->m_saved_match_len = d->m_saved_lit = 0; + d->m_adler32 = 1; + d->m_pIn_buf = NULL; + d->m_pOut_buf = NULL; + d->m_pIn_buf_size = NULL; + d->m_pOut_buf_size = NULL; + d->m_flush = TDEFL_NO_FLUSH; + d->m_pSrc = NULL; + d->m_src_buf_left = 0; + d->m_out_buf_ofs = 0; + if (!(flags & TDEFL_NONDETERMINISTIC_PARSING_FLAG)) + MZ_CLEAR_ARR(d->m_dict); + memset(&d->m_huff_count[0][0], 0, sizeof(d->m_huff_count[0][0]) * TDEFL_MAX_HUFF_SYMBOLS_0); + memset(&d->m_huff_count[1][0], 0, sizeof(d->m_huff_count[1][0]) * TDEFL_MAX_HUFF_SYMBOLS_1); + return TDEFL_STATUS_OKAY; + } + + tdefl_status tdefl_get_prev_return_status(tdefl_compressor *d) + { + return d->m_prev_return_status; + } + + mz_uint32 tdefl_get_adler32(tdefl_compressor *d) + { + return d->m_adler32; + } + + mz_bool tdefl_compress_mem_to_output(const void *pBuf, size_t buf_len, tdefl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags) + { + tdefl_compressor *pComp; + mz_bool succeeded; + if (((buf_len) && (!pBuf)) || (!pPut_buf_func)) + return MZ_FALSE; + pComp = (tdefl_compressor *)MZ_MALLOC(sizeof(tdefl_compressor)); + if (!pComp) + return MZ_FALSE; + succeeded = (tdefl_init(pComp, pPut_buf_func, pPut_buf_user, flags) == TDEFL_STATUS_OKAY); + succeeded = succeeded && (tdefl_compress_buffer(pComp, pBuf, buf_len, TDEFL_FINISH) == TDEFL_STATUS_DONE); + MZ_FREE(pComp); + return succeeded; + } + + typedef struct + { + size_t m_size, m_capacity; + mz_uint8 *m_pBuf; + mz_bool m_expandable; + } tdefl_output_buffer; + + static mz_bool tdefl_output_buffer_putter(const void *pBuf, int len, void *pUser) + { + tdefl_output_buffer *p = (tdefl_output_buffer *)pUser; + size_t new_size = p->m_size + len; + if (new_size > p->m_capacity) + { + size_t new_capacity = p->m_capacity; + mz_uint8 *pNew_buf; + if (!p->m_expandable) + return MZ_FALSE; + do + { + new_capacity = MZ_MAX(128U, new_capacity << 1U); + } while (new_size > new_capacity); + pNew_buf = (mz_uint8 *)MZ_REALLOC(p->m_pBuf, new_capacity); + if (!pNew_buf) + return MZ_FALSE; + p->m_pBuf = pNew_buf; + p->m_capacity = new_capacity; + } + memcpy((mz_uint8 *)p->m_pBuf + p->m_size, pBuf, len); + p->m_size = new_size; + return MZ_TRUE; + } + + void *tdefl_compress_mem_to_heap(const void *pSrc_buf, size_t src_buf_len, size_t *pOut_len, int flags) + { + tdefl_output_buffer out_buf; + MZ_CLEAR_OBJ(out_buf); + if (!pOut_len) + return MZ_FALSE; + else + *pOut_len = 0; + out_buf.m_expandable = MZ_TRUE; + if (!tdefl_compress_mem_to_output(pSrc_buf, src_buf_len, tdefl_output_buffer_putter, &out_buf, flags)) + return NULL; + *pOut_len = out_buf.m_size; + return out_buf.m_pBuf; + } + + size_t tdefl_compress_mem_to_mem(void *pOut_buf, size_t out_buf_len, const void *pSrc_buf, size_t src_buf_len, int flags) + { + tdefl_output_buffer out_buf; + MZ_CLEAR_OBJ(out_buf); + if (!pOut_buf) + return 0; + out_buf.m_pBuf = (mz_uint8 *)pOut_buf; + out_buf.m_capacity = out_buf_len; + if (!tdefl_compress_mem_to_output(pSrc_buf, src_buf_len, tdefl_output_buffer_putter, &out_buf, flags)) + return 0; + return out_buf.m_size; + } + + /* level may actually range from [0,10] (10 is a "hidden" max level, where we want a bit more compression and it's fine if throughput to fall off a cliff on some files). */ + mz_uint tdefl_create_comp_flags_from_zip_params(int level, int window_bits, int strategy) + { + mz_uint comp_flags = s_tdefl_num_probes[(level >= 0) ? MZ_MIN(10, level) : MZ_DEFAULT_LEVEL] | ((level <= 3) ? TDEFL_GREEDY_PARSING_FLAG : 0); + if (window_bits > 0) + comp_flags |= TDEFL_WRITE_ZLIB_HEADER; + + if (!level) + comp_flags |= TDEFL_FORCE_ALL_RAW_BLOCKS; + else if (strategy == MZ_FILTERED) + comp_flags |= TDEFL_FILTER_MATCHES; + else if (strategy == MZ_HUFFMAN_ONLY) + comp_flags &= ~TDEFL_MAX_PROBES_MASK; + else if (strategy == MZ_FIXED) + comp_flags |= TDEFL_FORCE_ALL_STATIC_BLOCKS; + else if (strategy == MZ_RLE) + comp_flags |= TDEFL_RLE_MATCHES; + + return comp_flags; + } + +#ifdef _MSC_VER +#pragma warning(push) +#pragma warning(disable : 4204) /* nonstandard extension used : non-constant aggregate initializer (also supported by GNU C and C99, so no big deal) */ +#endif + + /* Simple PNG writer function by Alex Evans, 2011. Released into the public domain: https://gist.github.com/908299, more context at + http://altdevblogaday.org/2011/04/06/a-smaller-jpg-encoder/. + This is actually a modification of Alex's original code so PNG files generated by this function pass pngcheck. */ + void *tdefl_write_image_to_png_file_in_memory_ex(const void *pImage, int w, int h, int num_chans, size_t *pLen_out, mz_uint level, mz_bool flip) + { + /* Using a local copy of this array here in case MINIZ_NO_ZLIB_APIS was defined. */ + static const mz_uint s_tdefl_png_num_probes[11] = { 0, 1, 6, 32, 16, 32, 128, 256, 512, 768, 1500 }; + tdefl_compressor *pComp = (tdefl_compressor *)MZ_MALLOC(sizeof(tdefl_compressor)); + tdefl_output_buffer out_buf; + int i, bpl = w * num_chans, y, z; + mz_uint32 c; + *pLen_out = 0; + if (!pComp) + return NULL; + MZ_CLEAR_OBJ(out_buf); + out_buf.m_expandable = MZ_TRUE; + out_buf.m_capacity = 57 + MZ_MAX(64, (1 + bpl) * h); + if (NULL == (out_buf.m_pBuf = (mz_uint8 *)MZ_MALLOC(out_buf.m_capacity))) + { + MZ_FREE(pComp); + return NULL; + } + /* write dummy header */ + for (z = 41; z; --z) + tdefl_output_buffer_putter(&z, 1, &out_buf); + /* compress image data */ + tdefl_init(pComp, tdefl_output_buffer_putter, &out_buf, s_tdefl_png_num_probes[MZ_MIN(10, level)] | TDEFL_WRITE_ZLIB_HEADER); + for (y = 0; y < h; ++y) + { + tdefl_compress_buffer(pComp, &z, 1, TDEFL_NO_FLUSH); + tdefl_compress_buffer(pComp, (mz_uint8 *)pImage + (flip ? (h - 1 - y) : y) * bpl, bpl, TDEFL_NO_FLUSH); + } + if (tdefl_compress_buffer(pComp, NULL, 0, TDEFL_FINISH) != TDEFL_STATUS_DONE) + { + MZ_FREE(pComp); + MZ_FREE(out_buf.m_pBuf); + return NULL; + } + /* write real header */ + *pLen_out = out_buf.m_size - 41; + { + static const mz_uint8 chans[] = { 0x00, 0x00, 0x04, 0x02, 0x06 }; + mz_uint8 pnghdr[41] = { 0x89, 0x50, 0x4e, 0x47, 0x0d, + 0x0a, 0x1a, 0x0a, 0x00, 0x00, + 0x00, 0x0d, 0x49, 0x48, 0x44, + 0x52, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x08, + 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x49, 0x44, 0x41, + 0x54 }; + pnghdr[18] = (mz_uint8)(w >> 8); + pnghdr[19] = (mz_uint8)w; + pnghdr[22] = (mz_uint8)(h >> 8); + pnghdr[23] = (mz_uint8)h; + pnghdr[25] = chans[num_chans]; + pnghdr[33] = (mz_uint8)(*pLen_out >> 24); + pnghdr[34] = (mz_uint8)(*pLen_out >> 16); + pnghdr[35] = (mz_uint8)(*pLen_out >> 8); + pnghdr[36] = (mz_uint8)*pLen_out; + c = (mz_uint32)mz_crc32(MZ_CRC32_INIT, pnghdr + 12, 17); + for (i = 0; i < 4; ++i, c <<= 8) + ((mz_uint8 *)(pnghdr + 29))[i] = (mz_uint8)(c >> 24); + memcpy(out_buf.m_pBuf, pnghdr, 41); + } + /* write footer (IDAT CRC-32, followed by IEND chunk) */ + if (!tdefl_output_buffer_putter("\0\0\0\0\0\0\0\0\x49\x45\x4e\x44\xae\x42\x60\x82", 16, &out_buf)) + { + *pLen_out = 0; + MZ_FREE(pComp); + MZ_FREE(out_buf.m_pBuf); + return NULL; + } + c = (mz_uint32)mz_crc32(MZ_CRC32_INIT, out_buf.m_pBuf + 41 - 4, *pLen_out + 4); + for (i = 0; i < 4; ++i, c <<= 8) + (out_buf.m_pBuf + out_buf.m_size - 16)[i] = (mz_uint8)(c >> 24); + /* compute final size of file, grab compressed data buffer and return */ + *pLen_out += 57; + MZ_FREE(pComp); + return out_buf.m_pBuf; + } + void *tdefl_write_image_to_png_file_in_memory(const void *pImage, int w, int h, int num_chans, size_t *pLen_out) + { + /* Level 6 corresponds to TDEFL_DEFAULT_MAX_PROBES or MZ_DEFAULT_LEVEL (but we can't depend on MZ_DEFAULT_LEVEL being available in case the zlib API's where #defined out) */ + return tdefl_write_image_to_png_file_in_memory_ex(pImage, w, h, num_chans, pLen_out, 6, MZ_FALSE); + } + +#ifndef MINIZ_NO_MALLOC + /* Allocate the tdefl_compressor and tinfl_decompressor structures in C so that */ + /* non-C language bindings to tdefL_ and tinfl_ API don't need to worry about */ + /* structure size and allocation mechanism. */ + tdefl_compressor *tdefl_compressor_alloc(void) + { + return (tdefl_compressor *)MZ_MALLOC(sizeof(tdefl_compressor)); + } + + void tdefl_compressor_free(tdefl_compressor *pComp) + { + MZ_FREE(pComp); + } +#endif + +#ifdef _MSC_VER +#pragma warning(pop) +#endif + +#ifdef __cplusplus +} +#endif + +#endif /*#ifndef MINIZ_NO_DEFLATE_APIS*/ diff --git a/src/helpers/miniz/miniz_tdef.h b/src/helpers/miniz/miniz_tdef.h new file mode 100644 index 000000000..7dc959bb0 --- /dev/null +++ b/src/helpers/miniz/miniz_tdef.h @@ -0,0 +1,199 @@ +#pragma once +#include "miniz_common.h" + +#ifndef MINIZ_NO_DEFLATE_APIS + +#ifdef __cplusplus +extern "C" +{ +#endif +/* ------------------- Low-level Compression API Definitions */ + +/* Set TDEFL_LESS_MEMORY to 1 to use less memory (compression will be slightly slower, and raw/dynamic blocks will be output more frequently). */ +#ifndef TDEFL_LESS_MEMORY +#define TDEFL_LESS_MEMORY 0 +#endif + + /* tdefl_init() compression flags logically OR'd together (low 12 bits contain the max. number of probes per dictionary search): */ + /* TDEFL_DEFAULT_MAX_PROBES: The compressor defaults to 128 dictionary probes per dictionary search. 0=Huffman only, 1=Huffman+LZ (fastest/crap compression), 4095=Huffman+LZ (slowest/best compression). */ + enum + { + TDEFL_HUFFMAN_ONLY = 0, + TDEFL_DEFAULT_MAX_PROBES = 128, + TDEFL_MAX_PROBES_MASK = 0xFFF + }; + + /* TDEFL_WRITE_ZLIB_HEADER: If set, the compressor outputs a zlib header before the deflate data, and the Adler-32 of the source data at the end. Otherwise, you'll get raw deflate data. */ + /* TDEFL_COMPUTE_ADLER32: Always compute the adler-32 of the input data (even when not writing zlib headers). */ + /* TDEFL_GREEDY_PARSING_FLAG: Set to use faster greedy parsing, instead of more efficient lazy parsing. */ + /* TDEFL_NONDETERMINISTIC_PARSING_FLAG: Enable to decrease the compressor's initialization time to the minimum, but the output may vary from run to run given the same input (depending on the contents of memory). */ + /* TDEFL_RLE_MATCHES: Only look for RLE matches (matches with a distance of 1) */ + /* TDEFL_FILTER_MATCHES: Discards matches <= 5 chars if enabled. */ + /* TDEFL_FORCE_ALL_STATIC_BLOCKS: Disable usage of optimized Huffman tables. */ + /* TDEFL_FORCE_ALL_RAW_BLOCKS: Only use raw (uncompressed) deflate blocks. */ + /* The low 12 bits are reserved to control the max # of hash probes per dictionary lookup (see TDEFL_MAX_PROBES_MASK). */ + enum + { + TDEFL_WRITE_ZLIB_HEADER = 0x01000, + TDEFL_COMPUTE_ADLER32 = 0x02000, + TDEFL_GREEDY_PARSING_FLAG = 0x04000, + TDEFL_NONDETERMINISTIC_PARSING_FLAG = 0x08000, + TDEFL_RLE_MATCHES = 0x10000, + TDEFL_FILTER_MATCHES = 0x20000, + TDEFL_FORCE_ALL_STATIC_BLOCKS = 0x40000, + TDEFL_FORCE_ALL_RAW_BLOCKS = 0x80000 + }; + + /* High level compression functions: */ + /* tdefl_compress_mem_to_heap() compresses a block in memory to a heap block allocated via malloc(). */ + /* On entry: */ + /* pSrc_buf, src_buf_len: Pointer and size of source block to compress. */ + /* flags: The max match finder probes (default is 128) logically OR'd against the above flags. Higher probes are slower but improve compression. */ + /* On return: */ + /* Function returns a pointer to the compressed data, or NULL on failure. */ + /* *pOut_len will be set to the compressed data's size, which could be larger than src_buf_len on uncompressible data. */ + /* The caller must free() the returned block when it's no longer needed. */ + MINIZ_EXPORT void *tdefl_compress_mem_to_heap(const void *pSrc_buf, size_t src_buf_len, size_t *pOut_len, int flags); + + /* tdefl_compress_mem_to_mem() compresses a block in memory to another block in memory. */ + /* Returns 0 on failure. */ + MINIZ_EXPORT size_t tdefl_compress_mem_to_mem(void *pOut_buf, size_t out_buf_len, const void *pSrc_buf, size_t src_buf_len, int flags); + + /* Compresses an image to a compressed PNG file in memory. */ + /* On entry: */ + /* pImage, w, h, and num_chans describe the image to compress. num_chans may be 1, 2, 3, or 4. */ + /* The image pitch in bytes per scanline will be w*num_chans. The leftmost pixel on the top scanline is stored first in memory. */ + /* level may range from [0,10], use MZ_NO_COMPRESSION, MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc. or a decent default is MZ_DEFAULT_LEVEL */ + /* If flip is true, the image will be flipped on the Y axis (useful for OpenGL apps). */ + /* On return: */ + /* Function returns a pointer to the compressed data, or NULL on failure. */ + /* *pLen_out will be set to the size of the PNG image file. */ + /* The caller must mz_free() the returned heap block (which will typically be larger than *pLen_out) when it's no longer needed. */ + MINIZ_EXPORT void *tdefl_write_image_to_png_file_in_memory_ex(const void *pImage, int w, int h, int num_chans, size_t *pLen_out, mz_uint level, mz_bool flip); + MINIZ_EXPORT void *tdefl_write_image_to_png_file_in_memory(const void *pImage, int w, int h, int num_chans, size_t *pLen_out); + + /* Output stream interface. The compressor uses this interface to write compressed data. It'll typically be called TDEFL_OUT_BUF_SIZE at a time. */ + typedef mz_bool (*tdefl_put_buf_func_ptr)(const void *pBuf, int len, void *pUser); + + /* tdefl_compress_mem_to_output() compresses a block to an output stream. The above helpers use this function internally. */ + MINIZ_EXPORT mz_bool tdefl_compress_mem_to_output(const void *pBuf, size_t buf_len, tdefl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags); + + enum + { + TDEFL_MAX_HUFF_TABLES = 3, + TDEFL_MAX_HUFF_SYMBOLS_0 = 288, + TDEFL_MAX_HUFF_SYMBOLS_1 = 32, + TDEFL_MAX_HUFF_SYMBOLS_2 = 19, + TDEFL_LZ_DICT_SIZE = 32768, + TDEFL_LZ_DICT_SIZE_MASK = TDEFL_LZ_DICT_SIZE - 1, + TDEFL_MIN_MATCH_LEN = 3, + TDEFL_MAX_MATCH_LEN = 258 + }; + +/* TDEFL_OUT_BUF_SIZE MUST be large enough to hold a single entire compressed output block (using static/fixed Huffman codes). */ +#if TDEFL_LESS_MEMORY + enum + { + TDEFL_LZ_CODE_BUF_SIZE = 24 * 1024, + TDEFL_OUT_BUF_SIZE = (TDEFL_LZ_CODE_BUF_SIZE * 13) / 10, + TDEFL_MAX_HUFF_SYMBOLS = 288, + TDEFL_LZ_HASH_BITS = 12, + TDEFL_LEVEL1_HASH_SIZE_MASK = 4095, + TDEFL_LZ_HASH_SHIFT = (TDEFL_LZ_HASH_BITS + 2) / 3, + TDEFL_LZ_HASH_SIZE = 1 << TDEFL_LZ_HASH_BITS + }; +#else +enum +{ + TDEFL_LZ_CODE_BUF_SIZE = 64 * 1024, + TDEFL_OUT_BUF_SIZE = (mz_uint)((TDEFL_LZ_CODE_BUF_SIZE * 13) / 10), + TDEFL_MAX_HUFF_SYMBOLS = 288, + TDEFL_LZ_HASH_BITS = 15, + TDEFL_LEVEL1_HASH_SIZE_MASK = 4095, + TDEFL_LZ_HASH_SHIFT = (TDEFL_LZ_HASH_BITS + 2) / 3, + TDEFL_LZ_HASH_SIZE = 1 << TDEFL_LZ_HASH_BITS +}; +#endif + + /* The low-level tdefl functions below may be used directly if the above helper functions aren't flexible enough. The low-level functions don't make any heap allocations, unlike the above helper functions. */ + typedef enum + { + TDEFL_STATUS_BAD_PARAM = -2, + TDEFL_STATUS_PUT_BUF_FAILED = -1, + TDEFL_STATUS_OKAY = 0, + TDEFL_STATUS_DONE = 1 + } tdefl_status; + + /* Must map to MZ_NO_FLUSH, MZ_SYNC_FLUSH, etc. enums */ + typedef enum + { + TDEFL_NO_FLUSH = 0, + TDEFL_SYNC_FLUSH = 2, + TDEFL_FULL_FLUSH = 3, + TDEFL_FINISH = 4 + } tdefl_flush; + + /* tdefl's compression state structure. */ + typedef struct + { + tdefl_put_buf_func_ptr m_pPut_buf_func; + void *m_pPut_buf_user; + mz_uint m_flags, m_max_probes[2]; + int m_greedy_parsing; + mz_uint m_adler32, m_lookahead_pos, m_lookahead_size, m_dict_size; + mz_uint8 *m_pLZ_code_buf, *m_pLZ_flags, *m_pOutput_buf, *m_pOutput_buf_end; + mz_uint m_num_flags_left, m_total_lz_bytes, m_lz_code_buf_dict_pos, m_bits_in, m_bit_buffer; + mz_uint m_saved_match_dist, m_saved_match_len, m_saved_lit, m_output_flush_ofs, m_output_flush_remaining, m_finished, m_block_index, m_wants_to_finish; + tdefl_status m_prev_return_status; + const void *m_pIn_buf; + void *m_pOut_buf; + size_t *m_pIn_buf_size, *m_pOut_buf_size; + tdefl_flush m_flush; + const mz_uint8 *m_pSrc; + size_t m_src_buf_left, m_out_buf_ofs; + mz_uint8 m_dict[TDEFL_LZ_DICT_SIZE + TDEFL_MAX_MATCH_LEN - 1]; + mz_uint16 m_huff_count[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS]; + mz_uint16 m_huff_codes[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS]; + mz_uint8 m_huff_code_sizes[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS]; + mz_uint8 m_lz_code_buf[TDEFL_LZ_CODE_BUF_SIZE]; + mz_uint16 m_next[TDEFL_LZ_DICT_SIZE]; + mz_uint16 m_hash[TDEFL_LZ_HASH_SIZE]; + mz_uint8 m_output_buf[TDEFL_OUT_BUF_SIZE]; + } tdefl_compressor; + + /* Initializes the compressor. */ + /* There is no corresponding deinit() function because the tdefl API's do not dynamically allocate memory. */ + /* pBut_buf_func: If NULL, output data will be supplied to the specified callback. In this case, the user should call the tdefl_compress_buffer() API for compression. */ + /* If pBut_buf_func is NULL the user should always call the tdefl_compress() API. */ + /* flags: See the above enums (TDEFL_HUFFMAN_ONLY, TDEFL_WRITE_ZLIB_HEADER, etc.) */ + MINIZ_EXPORT tdefl_status tdefl_init(tdefl_compressor *d, tdefl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags); + + /* Compresses a block of data, consuming as much of the specified input buffer as possible, and writing as much compressed data to the specified output buffer as possible. */ + MINIZ_EXPORT tdefl_status tdefl_compress(tdefl_compressor *d, const void *pIn_buf, size_t *pIn_buf_size, void *pOut_buf, size_t *pOut_buf_size, tdefl_flush flush); + + /* tdefl_compress_buffer() is only usable when the tdefl_init() is called with a non-NULL tdefl_put_buf_func_ptr. */ + /* tdefl_compress_buffer() always consumes the entire input buffer. */ + MINIZ_EXPORT tdefl_status tdefl_compress_buffer(tdefl_compressor *d, const void *pIn_buf, size_t in_buf_size, tdefl_flush flush); + + MINIZ_EXPORT tdefl_status tdefl_get_prev_return_status(tdefl_compressor *d); + MINIZ_EXPORT mz_uint32 tdefl_get_adler32(tdefl_compressor *d); + + /* Create tdefl_compress() flags given zlib-style compression parameters. */ + /* level may range from [0,10] (where 10 is absolute max compression, but may be much slower on some files) */ + /* window_bits may be -15 (raw deflate) or 15 (zlib) */ + /* strategy may be either MZ_DEFAULT_STRATEGY, MZ_FILTERED, MZ_HUFFMAN_ONLY, MZ_RLE, or MZ_FIXED */ + MINIZ_EXPORT mz_uint tdefl_create_comp_flags_from_zip_params(int level, int window_bits, int strategy); + +#ifndef MINIZ_NO_MALLOC + /* Allocate the tdefl_compressor structure in C so that */ + /* non-C language bindings to tdefl_ API don't need to worry about */ + /* structure size and allocation mechanism. */ + MINIZ_EXPORT tdefl_compressor *tdefl_compressor_alloc(void); + MINIZ_EXPORT void tdefl_compressor_free(tdefl_compressor *pComp); +#endif + +#ifdef __cplusplus +} +#endif + +#endif /*#ifndef MINIZ_NO_DEFLATE_APIS*/ diff --git a/src/helpers/miniz/miniz_tinfl.c b/src/helpers/miniz/miniz_tinfl.c new file mode 100644 index 000000000..3de465ff7 --- /dev/null +++ b/src/helpers/miniz/miniz_tinfl.c @@ -0,0 +1,770 @@ +/************************************************************************** + * + * Copyright 2013-2014 RAD Game Tools and Valve Software + * Copyright 2010-2014 Rich Geldreich and Tenacious Software LLC + * All Rights Reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + **************************************************************************/ + +#include "miniz.h" + +#ifndef MINIZ_NO_INFLATE_APIS + +#ifdef __cplusplus +extern "C" +{ +#endif + + /* ------------------- Low-level Decompression (completely independent from all compression API's) */ + +#define TINFL_MEMCPY(d, s, l) memcpy(d, s, l) +#define TINFL_MEMSET(p, c, l) memset(p, c, l) + +#define TINFL_CR_BEGIN \ + switch (r->m_state) \ + { \ + case 0: +#define TINFL_CR_RETURN(state_index, result) \ + do \ + { \ + status = result; \ + r->m_state = state_index; \ + goto common_exit; \ + case state_index:; \ + } \ + MZ_MACRO_END +#define TINFL_CR_RETURN_FOREVER(state_index, result) \ + do \ + { \ + for (;;) \ + { \ + TINFL_CR_RETURN(state_index, result); \ + } \ + } \ + MZ_MACRO_END +#define TINFL_CR_FINISH } + +#define TINFL_GET_BYTE(state_index, c) \ + do \ + { \ + while (pIn_buf_cur >= pIn_buf_end) \ + { \ + TINFL_CR_RETURN(state_index, (decomp_flags & TINFL_FLAG_HAS_MORE_INPUT) ? TINFL_STATUS_NEEDS_MORE_INPUT : TINFL_STATUS_FAILED_CANNOT_MAKE_PROGRESS); \ + } \ + c = *pIn_buf_cur++; \ + } \ + MZ_MACRO_END + +#define TINFL_NEED_BITS(state_index, n) \ + do \ + { \ + mz_uint c; \ + TINFL_GET_BYTE(state_index, c); \ + bit_buf |= (((tinfl_bit_buf_t)c) << num_bits); \ + num_bits += 8; \ + } while (num_bits < (mz_uint)(n)) +#define TINFL_SKIP_BITS(state_index, n) \ + do \ + { \ + if (num_bits < (mz_uint)(n)) \ + { \ + TINFL_NEED_BITS(state_index, n); \ + } \ + bit_buf >>= (n); \ + num_bits -= (n); \ + } \ + MZ_MACRO_END +#define TINFL_GET_BITS(state_index, b, n) \ + do \ + { \ + if (num_bits < (mz_uint)(n)) \ + { \ + TINFL_NEED_BITS(state_index, n); \ + } \ + b = bit_buf & ((1 << (n)) - 1); \ + bit_buf >>= (n); \ + num_bits -= (n); \ + } \ + MZ_MACRO_END + +/* TINFL_HUFF_BITBUF_FILL() is only used rarely, when the number of bytes remaining in the input buffer falls below 2. */ +/* It reads just enough bytes from the input stream that are needed to decode the next Huffman code (and absolutely no more). It works by trying to fully decode a */ +/* Huffman code by using whatever bits are currently present in the bit buffer. If this fails, it reads another byte, and tries again until it succeeds or until the */ +/* bit buffer contains >=15 bits (deflate's max. Huffman code size). */ +#define TINFL_HUFF_BITBUF_FILL(state_index, pLookUp, pTree) \ + do \ + { \ + temp = pLookUp[bit_buf & (TINFL_FAST_LOOKUP_SIZE - 1)]; \ + if (temp >= 0) \ + { \ + code_len = temp >> 9; \ + if ((code_len) && (num_bits >= code_len)) \ + break; \ + } \ + else if (num_bits > TINFL_FAST_LOOKUP_BITS) \ + { \ + code_len = TINFL_FAST_LOOKUP_BITS; \ + do \ + { \ + temp = pTree[~temp + ((bit_buf >> code_len++) & 1)]; \ + } while ((temp < 0) && (num_bits >= (code_len + 1))); \ + if (temp >= 0) \ + break; \ + } \ + TINFL_GET_BYTE(state_index, c); \ + bit_buf |= (((tinfl_bit_buf_t)c) << num_bits); \ + num_bits += 8; \ + } while (num_bits < 15); + +/* TINFL_HUFF_DECODE() decodes the next Huffman coded symbol. It's more complex than you would initially expect because the zlib API expects the decompressor to never read */ +/* beyond the final byte of the deflate stream. (In other words, when this macro wants to read another byte from the input, it REALLY needs another byte in order to fully */ +/* decode the next Huffman code.) Handling this properly is particularly important on raw deflate (non-zlib) streams, which aren't followed by a byte aligned adler-32. */ +/* The slow path is only executed at the very end of the input buffer. */ +/* v1.16: The original macro handled the case at the very end of the passed-in input buffer, but we also need to handle the case where the user passes in 1+zillion bytes */ +/* following the deflate data and our non-conservative read-ahead path won't kick in here on this code. This is much trickier. */ +#define TINFL_HUFF_DECODE(state_index, sym, pLookUp, pTree) \ + do \ + { \ + int temp; \ + mz_uint code_len, c; \ + if (num_bits < 15) \ + { \ + if ((pIn_buf_end - pIn_buf_cur) < 2) \ + { \ + TINFL_HUFF_BITBUF_FILL(state_index, pLookUp, pTree); \ + } \ + else \ + { \ + bit_buf |= (((tinfl_bit_buf_t)pIn_buf_cur[0]) << num_bits) | (((tinfl_bit_buf_t)pIn_buf_cur[1]) << (num_bits + 8)); \ + pIn_buf_cur += 2; \ + num_bits += 16; \ + } \ + } \ + if ((temp = pLookUp[bit_buf & (TINFL_FAST_LOOKUP_SIZE - 1)]) >= 0) \ + code_len = temp >> 9, temp &= 511; \ + else \ + { \ + code_len = TINFL_FAST_LOOKUP_BITS; \ + do \ + { \ + temp = pTree[~temp + ((bit_buf >> code_len++) & 1)]; \ + } while (temp < 0); \ + } \ + sym = temp; \ + bit_buf >>= code_len; \ + num_bits -= code_len; \ + } \ + MZ_MACRO_END + + static void tinfl_clear_tree(tinfl_decompressor *r) + { + if (r->m_type == 0) + MZ_CLEAR_ARR(r->m_tree_0); + else if (r->m_type == 1) + MZ_CLEAR_ARR(r->m_tree_1); + else + MZ_CLEAR_ARR(r->m_tree_2); + } + + tinfl_status tinfl_decompress(tinfl_decompressor *r, const mz_uint8 *pIn_buf_next, size_t *pIn_buf_size, mz_uint8 *pOut_buf_start, mz_uint8 *pOut_buf_next, size_t *pOut_buf_size, const mz_uint32 decomp_flags) + { + static const mz_uint16 s_length_base[31] = { 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258, 0, 0 }; + static const mz_uint8 s_length_extra[31] = { 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0, 0, 0 }; + static const mz_uint16 s_dist_base[32] = { 1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577, 0, 0 }; + static const mz_uint8 s_dist_extra[32] = { 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13 }; + static const mz_uint8 s_length_dezigzag[19] = { 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 }; + static const mz_uint16 s_min_table_sizes[3] = { 257, 1, 4 }; + + mz_int16 *pTrees[3]; + mz_uint8 *pCode_sizes[3]; + + tinfl_status status = TINFL_STATUS_FAILED; + mz_uint32 num_bits, dist, counter, num_extra; + tinfl_bit_buf_t bit_buf; + const mz_uint8 *pIn_buf_cur = pIn_buf_next, *const pIn_buf_end = pIn_buf_next + *pIn_buf_size; + mz_uint8 *pOut_buf_cur = pOut_buf_next, *const pOut_buf_end = pOut_buf_next ? pOut_buf_next + *pOut_buf_size : NULL; + size_t out_buf_size_mask = (decomp_flags & TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF) ? (size_t)-1 : ((pOut_buf_next - pOut_buf_start) + *pOut_buf_size) - 1, dist_from_out_buf_start; + + /* Ensure the output buffer's size is a power of 2, unless the output buffer is large enough to hold the entire output file (in which case it doesn't matter). */ + if (((out_buf_size_mask + 1) & out_buf_size_mask) || (pOut_buf_next < pOut_buf_start)) + { + *pIn_buf_size = *pOut_buf_size = 0; + return TINFL_STATUS_BAD_PARAM; + } + + pTrees[0] = r->m_tree_0; + pTrees[1] = r->m_tree_1; + pTrees[2] = r->m_tree_2; + pCode_sizes[0] = r->m_code_size_0; + pCode_sizes[1] = r->m_code_size_1; + pCode_sizes[2] = r->m_code_size_2; + + num_bits = r->m_num_bits; + bit_buf = r->m_bit_buf; + dist = r->m_dist; + counter = r->m_counter; + num_extra = r->m_num_extra; + dist_from_out_buf_start = r->m_dist_from_out_buf_start; + TINFL_CR_BEGIN + + bit_buf = num_bits = dist = counter = num_extra = r->m_zhdr0 = r->m_zhdr1 = 0; + r->m_z_adler32 = r->m_check_adler32 = 1; + if (decomp_flags & TINFL_FLAG_PARSE_ZLIB_HEADER) + { + TINFL_GET_BYTE(1, r->m_zhdr0); + TINFL_GET_BYTE(2, r->m_zhdr1); + counter = (((r->m_zhdr0 * 256 + r->m_zhdr1) % 31 != 0) || (r->m_zhdr1 & 32) || ((r->m_zhdr0 & 15) != 8)); + if (!(decomp_flags & TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF)) + counter |= (((1U << (8U + (r->m_zhdr0 >> 4))) > 32768U) || ((out_buf_size_mask + 1) < (size_t)((size_t)1 << (8U + (r->m_zhdr0 >> 4))))); + if (counter) + { + TINFL_CR_RETURN_FOREVER(36, TINFL_STATUS_FAILED); + } + } + + do + { + TINFL_GET_BITS(3, r->m_final, 3); + r->m_type = r->m_final >> 1; + if (r->m_type == 0) + { + TINFL_SKIP_BITS(5, num_bits & 7); + for (counter = 0; counter < 4; ++counter) + { + if (num_bits) + TINFL_GET_BITS(6, r->m_raw_header[counter], 8); + else + TINFL_GET_BYTE(7, r->m_raw_header[counter]); + } + if ((counter = (r->m_raw_header[0] | (r->m_raw_header[1] << 8))) != (mz_uint)(0xFFFF ^ (r->m_raw_header[2] | (r->m_raw_header[3] << 8)))) + { + TINFL_CR_RETURN_FOREVER(39, TINFL_STATUS_FAILED); + } + while ((counter) && (num_bits)) + { + TINFL_GET_BITS(51, dist, 8); + while (pOut_buf_cur >= pOut_buf_end) + { + TINFL_CR_RETURN(52, TINFL_STATUS_HAS_MORE_OUTPUT); + } + *pOut_buf_cur++ = (mz_uint8)dist; + counter--; + } + while (counter) + { + size_t n; + while (pOut_buf_cur >= pOut_buf_end) + { + TINFL_CR_RETURN(9, TINFL_STATUS_HAS_MORE_OUTPUT); + } + while (pIn_buf_cur >= pIn_buf_end) + { + TINFL_CR_RETURN(38, (decomp_flags & TINFL_FLAG_HAS_MORE_INPUT) ? TINFL_STATUS_NEEDS_MORE_INPUT : TINFL_STATUS_FAILED_CANNOT_MAKE_PROGRESS); + } + n = MZ_MIN(MZ_MIN((size_t)(pOut_buf_end - pOut_buf_cur), (size_t)(pIn_buf_end - pIn_buf_cur)), counter); + TINFL_MEMCPY(pOut_buf_cur, pIn_buf_cur, n); + pIn_buf_cur += n; + pOut_buf_cur += n; + counter -= (mz_uint)n; + } + } + else if (r->m_type == 3) + { + TINFL_CR_RETURN_FOREVER(10, TINFL_STATUS_FAILED); + } + else + { + if (r->m_type == 1) + { + mz_uint8 *p = r->m_code_size_0; + mz_uint i; + r->m_table_sizes[0] = 288; + r->m_table_sizes[1] = 32; + TINFL_MEMSET(r->m_code_size_1, 5, 32); + for (i = 0; i <= 143; ++i) + *p++ = 8; + for (; i <= 255; ++i) + *p++ = 9; + for (; i <= 279; ++i) + *p++ = 7; + for (; i <= 287; ++i) + *p++ = 8; + } + else + { + for (counter = 0; counter < 3; counter++) + { + TINFL_GET_BITS(11, r->m_table_sizes[counter], "\05\05\04"[counter]); + r->m_table_sizes[counter] += s_min_table_sizes[counter]; + } + MZ_CLEAR_ARR(r->m_code_size_2); + for (counter = 0; counter < r->m_table_sizes[2]; counter++) + { + mz_uint s; + TINFL_GET_BITS(14, s, 3); + r->m_code_size_2[s_length_dezigzag[counter]] = (mz_uint8)s; + } + r->m_table_sizes[2] = 19; + } + for (; (int)r->m_type >= 0; r->m_type--) + { + int tree_next, tree_cur; + mz_int16 *pLookUp; + mz_int16 *pTree; + mz_uint8 *pCode_size; + mz_uint i, j, used_syms, total, sym_index, next_code[17], total_syms[16]; + pLookUp = r->m_look_up[r->m_type]; + pTree = pTrees[r->m_type]; + pCode_size = pCode_sizes[r->m_type]; + MZ_CLEAR_ARR(total_syms); + TINFL_MEMSET(pLookUp, 0, sizeof(r->m_look_up[0])); + tinfl_clear_tree(r); + for (i = 0; i < r->m_table_sizes[r->m_type]; ++i) + total_syms[pCode_size[i]]++; + used_syms = 0, total = 0; + next_code[0] = next_code[1] = 0; + for (i = 1; i <= 15; ++i) + { + used_syms += total_syms[i]; + next_code[i + 1] = (total = ((total + total_syms[i]) << 1)); + } + if ((65536 != total) && (used_syms > 1)) + { + TINFL_CR_RETURN_FOREVER(35, TINFL_STATUS_FAILED); + } + for (tree_next = -1, sym_index = 0; sym_index < r->m_table_sizes[r->m_type]; ++sym_index) + { + mz_uint rev_code = 0, l, cur_code, code_size = pCode_size[sym_index]; + if (!code_size) + continue; + cur_code = next_code[code_size]++; + for (l = code_size; l > 0; l--, cur_code >>= 1) + rev_code = (rev_code << 1) | (cur_code & 1); + if (code_size <= TINFL_FAST_LOOKUP_BITS) + { + mz_int16 k = (mz_int16)((code_size << 9) | sym_index); + while (rev_code < TINFL_FAST_LOOKUP_SIZE) + { + pLookUp[rev_code] = k; + rev_code += (1 << code_size); + } + continue; + } + if (0 == (tree_cur = pLookUp[rev_code & (TINFL_FAST_LOOKUP_SIZE - 1)])) + { + pLookUp[rev_code & (TINFL_FAST_LOOKUP_SIZE - 1)] = (mz_int16)tree_next; + tree_cur = tree_next; + tree_next -= 2; + } + rev_code >>= (TINFL_FAST_LOOKUP_BITS - 1); + for (j = code_size; j > (TINFL_FAST_LOOKUP_BITS + 1); j--) + { + tree_cur -= ((rev_code >>= 1) & 1); + if (!pTree[-tree_cur - 1]) + { + pTree[-tree_cur - 1] = (mz_int16)tree_next; + tree_cur = tree_next; + tree_next -= 2; + } + else + tree_cur = pTree[-tree_cur - 1]; + } + tree_cur -= ((rev_code >>= 1) & 1); + pTree[-tree_cur - 1] = (mz_int16)sym_index; + } + if (r->m_type == 2) + { + for (counter = 0; counter < (r->m_table_sizes[0] + r->m_table_sizes[1]);) + { + mz_uint s; + TINFL_HUFF_DECODE(16, dist, r->m_look_up[2], r->m_tree_2); + if (dist < 16) + { + r->m_len_codes[counter++] = (mz_uint8)dist; + continue; + } + if ((dist == 16) && (!counter)) + { + TINFL_CR_RETURN_FOREVER(17, TINFL_STATUS_FAILED); + } + num_extra = "\02\03\07"[dist - 16]; + TINFL_GET_BITS(18, s, num_extra); + s += "\03\03\013"[dist - 16]; + TINFL_MEMSET(r->m_len_codes + counter, (dist == 16) ? r->m_len_codes[counter - 1] : 0, s); + counter += s; + } + if ((r->m_table_sizes[0] + r->m_table_sizes[1]) != counter) + { + TINFL_CR_RETURN_FOREVER(21, TINFL_STATUS_FAILED); + } + TINFL_MEMCPY(r->m_code_size_0, r->m_len_codes, r->m_table_sizes[0]); + TINFL_MEMCPY(r->m_code_size_1, r->m_len_codes + r->m_table_sizes[0], r->m_table_sizes[1]); + } + } + for (;;) + { + mz_uint8 *pSrc; + for (;;) + { + if (((pIn_buf_end - pIn_buf_cur) < 4) || ((pOut_buf_end - pOut_buf_cur) < 2)) + { + TINFL_HUFF_DECODE(23, counter, r->m_look_up[0], r->m_tree_0); + if (counter >= 256) + break; + while (pOut_buf_cur >= pOut_buf_end) + { + TINFL_CR_RETURN(24, TINFL_STATUS_HAS_MORE_OUTPUT); + } + *pOut_buf_cur++ = (mz_uint8)counter; + } + else + { + int sym2; + mz_uint code_len; +#if TINFL_USE_64BIT_BITBUF + if (num_bits < 30) + { + bit_buf |= (((tinfl_bit_buf_t)MZ_READ_LE32(pIn_buf_cur)) << num_bits); + pIn_buf_cur += 4; + num_bits += 32; + } +#else + if (num_bits < 15) + { + bit_buf |= (((tinfl_bit_buf_t)MZ_READ_LE16(pIn_buf_cur)) << num_bits); + pIn_buf_cur += 2; + num_bits += 16; + } +#endif + if ((sym2 = r->m_look_up[0][bit_buf & (TINFL_FAST_LOOKUP_SIZE - 1)]) >= 0) + code_len = sym2 >> 9; + else + { + code_len = TINFL_FAST_LOOKUP_BITS; + do + { + sym2 = r->m_tree_0[~sym2 + ((bit_buf >> code_len++) & 1)]; + } while (sym2 < 0); + } + counter = sym2; + bit_buf >>= code_len; + num_bits -= code_len; + if (counter & 256) + break; + +#if !TINFL_USE_64BIT_BITBUF + if (num_bits < 15) + { + bit_buf |= (((tinfl_bit_buf_t)MZ_READ_LE16(pIn_buf_cur)) << num_bits); + pIn_buf_cur += 2; + num_bits += 16; + } +#endif + if ((sym2 = r->m_look_up[0][bit_buf & (TINFL_FAST_LOOKUP_SIZE - 1)]) >= 0) + code_len = sym2 >> 9; + else + { + code_len = TINFL_FAST_LOOKUP_BITS; + do + { + sym2 = r->m_tree_0[~sym2 + ((bit_buf >> code_len++) & 1)]; + } while (sym2 < 0); + } + bit_buf >>= code_len; + num_bits -= code_len; + + pOut_buf_cur[0] = (mz_uint8)counter; + if (sym2 & 256) + { + pOut_buf_cur++; + counter = sym2; + break; + } + pOut_buf_cur[1] = (mz_uint8)sym2; + pOut_buf_cur += 2; + } + } + if ((counter &= 511) == 256) + break; + + num_extra = s_length_extra[counter - 257]; + counter = s_length_base[counter - 257]; + if (num_extra) + { + mz_uint extra_bits; + TINFL_GET_BITS(25, extra_bits, num_extra); + counter += extra_bits; + } + + TINFL_HUFF_DECODE(26, dist, r->m_look_up[1], r->m_tree_1); + num_extra = s_dist_extra[dist]; + dist = s_dist_base[dist]; + if (num_extra) + { + mz_uint extra_bits; + TINFL_GET_BITS(27, extra_bits, num_extra); + dist += extra_bits; + } + + dist_from_out_buf_start = pOut_buf_cur - pOut_buf_start; + if ((dist == 0 || dist > dist_from_out_buf_start || dist_from_out_buf_start == 0) && (decomp_flags & TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF)) + { + TINFL_CR_RETURN_FOREVER(37, TINFL_STATUS_FAILED); + } + + pSrc = pOut_buf_start + ((dist_from_out_buf_start - dist) & out_buf_size_mask); + + if ((MZ_MAX(pOut_buf_cur, pSrc) + counter) > pOut_buf_end) + { + while (counter--) + { + while (pOut_buf_cur >= pOut_buf_end) + { + TINFL_CR_RETURN(53, TINFL_STATUS_HAS_MORE_OUTPUT); + } + *pOut_buf_cur++ = pOut_buf_start[(dist_from_out_buf_start++ - dist) & out_buf_size_mask]; + } + continue; + } +#if MINIZ_USE_UNALIGNED_LOADS_AND_STORES + else if ((counter >= 9) && (counter <= dist)) + { + const mz_uint8 *pSrc_end = pSrc + (counter & ~7); + do + { +#ifdef MINIZ_UNALIGNED_USE_MEMCPY + memcpy(pOut_buf_cur, pSrc, sizeof(mz_uint32) * 2); +#else + ((mz_uint32 *)pOut_buf_cur)[0] = ((const mz_uint32 *)pSrc)[0]; + ((mz_uint32 *)pOut_buf_cur)[1] = ((const mz_uint32 *)pSrc)[1]; +#endif + pOut_buf_cur += 8; + } while ((pSrc += 8) < pSrc_end); + if ((counter &= 7) < 3) + { + if (counter) + { + pOut_buf_cur[0] = pSrc[0]; + if (counter > 1) + pOut_buf_cur[1] = pSrc[1]; + pOut_buf_cur += counter; + } + continue; + } + } +#endif + while (counter > 2) + { + pOut_buf_cur[0] = pSrc[0]; + pOut_buf_cur[1] = pSrc[1]; + pOut_buf_cur[2] = pSrc[2]; + pOut_buf_cur += 3; + pSrc += 3; + counter -= 3; + } + if (counter > 0) + { + pOut_buf_cur[0] = pSrc[0]; + if (counter > 1) + pOut_buf_cur[1] = pSrc[1]; + pOut_buf_cur += counter; + } + } + } + } while (!(r->m_final & 1)); + + /* Ensure byte alignment and put back any bytes from the bitbuf if we've looked ahead too far on gzip, or other Deflate streams followed by arbitrary data. */ + /* I'm being super conservative here. A number of simplifications can be made to the byte alignment part, and the Adler32 check shouldn't ever need to worry about reading from the bitbuf now. */ + TINFL_SKIP_BITS(32, num_bits & 7); + while ((pIn_buf_cur > pIn_buf_next) && (num_bits >= 8)) + { + --pIn_buf_cur; + num_bits -= 8; + } + bit_buf &= ~(~(tinfl_bit_buf_t)0 << num_bits); + MZ_ASSERT(!num_bits); /* if this assert fires then we've read beyond the end of non-deflate/zlib streams with following data (such as gzip streams). */ + + if (decomp_flags & TINFL_FLAG_PARSE_ZLIB_HEADER) + { + for (counter = 0; counter < 4; ++counter) + { + mz_uint s; + if (num_bits) + TINFL_GET_BITS(41, s, 8); + else + TINFL_GET_BYTE(42, s); + r->m_z_adler32 = (r->m_z_adler32 << 8) | s; + } + } + TINFL_CR_RETURN_FOREVER(34, TINFL_STATUS_DONE); + + TINFL_CR_FINISH + + common_exit: + /* As long as we aren't telling the caller that we NEED more input to make forward progress: */ + /* Put back any bytes from the bitbuf in case we've looked ahead too far on gzip, or other Deflate streams followed by arbitrary data. */ + /* We need to be very careful here to NOT push back any bytes we definitely know we need to make forward progress, though, or we'll lock the caller up into an inf loop. */ + if ((status != TINFL_STATUS_NEEDS_MORE_INPUT) && (status != TINFL_STATUS_FAILED_CANNOT_MAKE_PROGRESS)) + { + while ((pIn_buf_cur > pIn_buf_next) && (num_bits >= 8)) + { + --pIn_buf_cur; + num_bits -= 8; + } + } + r->m_num_bits = num_bits; + r->m_bit_buf = bit_buf & ~(~(tinfl_bit_buf_t)0 << num_bits); + r->m_dist = dist; + r->m_counter = counter; + r->m_num_extra = num_extra; + r->m_dist_from_out_buf_start = dist_from_out_buf_start; + *pIn_buf_size = pIn_buf_cur - pIn_buf_next; + *pOut_buf_size = pOut_buf_cur - pOut_buf_next; + if ((decomp_flags & (TINFL_FLAG_PARSE_ZLIB_HEADER | TINFL_FLAG_COMPUTE_ADLER32)) && (status >= 0)) + { + const mz_uint8 *ptr = pOut_buf_next; + size_t buf_len = *pOut_buf_size; + mz_uint32 i, s1 = r->m_check_adler32 & 0xffff, s2 = r->m_check_adler32 >> 16; + size_t block_len = buf_len % 5552; + while (buf_len) + { + for (i = 0; i + 7 < block_len; i += 8, ptr += 8) + { + s1 += ptr[0], s2 += s1; + s1 += ptr[1], s2 += s1; + s1 += ptr[2], s2 += s1; + s1 += ptr[3], s2 += s1; + s1 += ptr[4], s2 += s1; + s1 += ptr[5], s2 += s1; + s1 += ptr[6], s2 += s1; + s1 += ptr[7], s2 += s1; + } + for (; i < block_len; ++i) + s1 += *ptr++, s2 += s1; + s1 %= 65521U, s2 %= 65521U; + buf_len -= block_len; + block_len = 5552; + } + r->m_check_adler32 = (s2 << 16) + s1; + if ((status == TINFL_STATUS_DONE) && (decomp_flags & TINFL_FLAG_PARSE_ZLIB_HEADER) && (r->m_check_adler32 != r->m_z_adler32)) + status = TINFL_STATUS_ADLER32_MISMATCH; + } + return status; + } + + /* Higher level helper functions. */ + void *tinfl_decompress_mem_to_heap(const void *pSrc_buf, size_t src_buf_len, size_t *pOut_len, int flags) + { + tinfl_decompressor decomp; + void *pBuf = NULL, *pNew_buf; + size_t src_buf_ofs = 0, out_buf_capacity = 0; + *pOut_len = 0; + tinfl_init(&decomp); + for (;;) + { + size_t src_buf_size = src_buf_len - src_buf_ofs, dst_buf_size = out_buf_capacity - *pOut_len, new_out_buf_capacity; + tinfl_status status = tinfl_decompress(&decomp, (const mz_uint8 *)pSrc_buf + src_buf_ofs, &src_buf_size, (mz_uint8 *)pBuf, pBuf ? (mz_uint8 *)pBuf + *pOut_len : NULL, &dst_buf_size, + (flags & ~TINFL_FLAG_HAS_MORE_INPUT) | TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF); + if ((status < 0) || (status == TINFL_STATUS_NEEDS_MORE_INPUT)) + { + MZ_FREE(pBuf); + *pOut_len = 0; + return NULL; + } + src_buf_ofs += src_buf_size; + *pOut_len += dst_buf_size; + if (status == TINFL_STATUS_DONE) + break; + new_out_buf_capacity = out_buf_capacity * 2; + if (new_out_buf_capacity < 128) + new_out_buf_capacity = 128; + pNew_buf = MZ_REALLOC(pBuf, new_out_buf_capacity); + if (!pNew_buf) + { + MZ_FREE(pBuf); + *pOut_len = 0; + return NULL; + } + pBuf = pNew_buf; + out_buf_capacity = new_out_buf_capacity; + } + return pBuf; + } + + size_t tinfl_decompress_mem_to_mem(void *pOut_buf, size_t out_buf_len, const void *pSrc_buf, size_t src_buf_len, int flags) + { + tinfl_decompressor decomp; + tinfl_status status; + tinfl_init(&decomp); + status = tinfl_decompress(&decomp, (const mz_uint8 *)pSrc_buf, &src_buf_len, (mz_uint8 *)pOut_buf, (mz_uint8 *)pOut_buf, &out_buf_len, (flags & ~TINFL_FLAG_HAS_MORE_INPUT) | TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF); + return (status != TINFL_STATUS_DONE) ? TINFL_DECOMPRESS_MEM_TO_MEM_FAILED : out_buf_len; + } + + int tinfl_decompress_mem_to_callback(const void *pIn_buf, size_t *pIn_buf_size, tinfl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags) + { + int result = 0; + tinfl_decompressor decomp; + mz_uint8 *pDict = (mz_uint8 *)MZ_MALLOC(TINFL_LZ_DICT_SIZE); + size_t in_buf_ofs = 0, dict_ofs = 0; + if (!pDict) + return TINFL_STATUS_FAILED; + memset(pDict, 0, TINFL_LZ_DICT_SIZE); + tinfl_init(&decomp); + for (;;) + { + size_t in_buf_size = *pIn_buf_size - in_buf_ofs, dst_buf_size = TINFL_LZ_DICT_SIZE - dict_ofs; + tinfl_status status = tinfl_decompress(&decomp, (const mz_uint8 *)pIn_buf + in_buf_ofs, &in_buf_size, pDict, pDict + dict_ofs, &dst_buf_size, + (flags & ~(TINFL_FLAG_HAS_MORE_INPUT | TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF))); + in_buf_ofs += in_buf_size; + if ((dst_buf_size) && (!(*pPut_buf_func)(pDict + dict_ofs, (int)dst_buf_size, pPut_buf_user))) + break; + if (status != TINFL_STATUS_HAS_MORE_OUTPUT) + { + result = (status == TINFL_STATUS_DONE); + break; + } + dict_ofs = (dict_ofs + dst_buf_size) & (TINFL_LZ_DICT_SIZE - 1); + } + MZ_FREE(pDict); + *pIn_buf_size = in_buf_ofs; + return result; + } + +#ifndef MINIZ_NO_MALLOC + tinfl_decompressor *tinfl_decompressor_alloc(void) + { + tinfl_decompressor *pDecomp = (tinfl_decompressor *)MZ_MALLOC(sizeof(tinfl_decompressor)); + if (pDecomp) + tinfl_init(pDecomp); + return pDecomp; + } + + void tinfl_decompressor_free(tinfl_decompressor *pDecomp) + { + MZ_FREE(pDecomp); + } +#endif + +#ifdef __cplusplus +} +#endif + +#endif /*#ifndef MINIZ_NO_INFLATE_APIS*/ diff --git a/src/helpers/miniz/miniz_tinfl.h b/src/helpers/miniz/miniz_tinfl.h new file mode 100644 index 000000000..7edca607c --- /dev/null +++ b/src/helpers/miniz/miniz_tinfl.h @@ -0,0 +1,150 @@ +#pragma once +#include "miniz_common.h" +/* ------------------- Low-level Decompression API Definitions */ + +#ifndef MINIZ_NO_INFLATE_APIS + +#ifdef __cplusplus +extern "C" +{ +#endif + /* Decompression flags used by tinfl_decompress(). */ + /* TINFL_FLAG_PARSE_ZLIB_HEADER: If set, the input has a valid zlib header and ends with an adler32 checksum (it's a valid zlib stream). Otherwise, the input is a raw deflate stream. */ + /* TINFL_FLAG_HAS_MORE_INPUT: If set, there are more input bytes available beyond the end of the supplied input buffer. If clear, the input buffer contains all remaining input. */ + /* TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF: If set, the output buffer is large enough to hold the entire decompressed stream. If clear, the output buffer is at least the size of the dictionary (typically 32KB). */ + /* TINFL_FLAG_COMPUTE_ADLER32: Force adler-32 checksum computation of the decompressed bytes. */ + enum + { + TINFL_FLAG_PARSE_ZLIB_HEADER = 1, + TINFL_FLAG_HAS_MORE_INPUT = 2, + TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF = 4, + TINFL_FLAG_COMPUTE_ADLER32 = 8 + }; + + /* High level decompression functions: */ + /* tinfl_decompress_mem_to_heap() decompresses a block in memory to a heap block allocated via malloc(). */ + /* On entry: */ + /* pSrc_buf, src_buf_len: Pointer and size of the Deflate or zlib source data to decompress. */ + /* On return: */ + /* Function returns a pointer to the decompressed data, or NULL on failure. */ + /* *pOut_len will be set to the decompressed data's size, which could be larger than src_buf_len on uncompressible data. */ + /* The caller must call mz_free() on the returned block when it's no longer needed. */ + MINIZ_EXPORT void *tinfl_decompress_mem_to_heap(const void *pSrc_buf, size_t src_buf_len, size_t *pOut_len, int flags); + +/* tinfl_decompress_mem_to_mem() decompresses a block in memory to another block in memory. */ +/* Returns TINFL_DECOMPRESS_MEM_TO_MEM_FAILED on failure, or the number of bytes written on success. */ +#define TINFL_DECOMPRESS_MEM_TO_MEM_FAILED ((size_t)(-1)) + MINIZ_EXPORT size_t tinfl_decompress_mem_to_mem(void *pOut_buf, size_t out_buf_len, const void *pSrc_buf, size_t src_buf_len, int flags); + + /* tinfl_decompress_mem_to_callback() decompresses a block in memory to an internal 32KB buffer, and a user provided callback function will be called to flush the buffer. */ + /* Returns 1 on success or 0 on failure. */ + typedef int (*tinfl_put_buf_func_ptr)(const void *pBuf, int len, void *pUser); + MINIZ_EXPORT int tinfl_decompress_mem_to_callback(const void *pIn_buf, size_t *pIn_buf_size, tinfl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags); + + struct tinfl_decompressor_tag; + typedef struct tinfl_decompressor_tag tinfl_decompressor; + +#ifndef MINIZ_NO_MALLOC + /* Allocate the tinfl_decompressor structure in C so that */ + /* non-C language bindings to tinfl_ API don't need to worry about */ + /* structure size and allocation mechanism. */ + MINIZ_EXPORT tinfl_decompressor *tinfl_decompressor_alloc(void); + MINIZ_EXPORT void tinfl_decompressor_free(tinfl_decompressor *pDecomp); +#endif + +/* Max size of LZ dictionary. */ +#define TINFL_LZ_DICT_SIZE 32768 + + /* Return status. */ + typedef enum + { + /* This flags indicates the inflator needs 1 or more input bytes to make forward progress, but the caller is indicating that no more are available. The compressed data */ + /* is probably corrupted. If you call the inflator again with more bytes it'll try to continue processing the input but this is a BAD sign (either the data is corrupted or you called it incorrectly). */ + /* If you call it again with no input you'll just get TINFL_STATUS_FAILED_CANNOT_MAKE_PROGRESS again. */ + TINFL_STATUS_FAILED_CANNOT_MAKE_PROGRESS = -4, + + /* This flag indicates that one or more of the input parameters was obviously bogus. (You can try calling it again, but if you get this error the calling code is wrong.) */ + TINFL_STATUS_BAD_PARAM = -3, + + /* This flags indicate the inflator is finished but the adler32 check of the uncompressed data didn't match. If you call it again it'll return TINFL_STATUS_DONE. */ + TINFL_STATUS_ADLER32_MISMATCH = -2, + + /* This flags indicate the inflator has somehow failed (bad code, corrupted input, etc.). If you call it again without resetting via tinfl_init() it it'll just keep on returning the same status failure code. */ + TINFL_STATUS_FAILED = -1, + + /* Any status code less than TINFL_STATUS_DONE must indicate a failure. */ + + /* This flag indicates the inflator has returned every byte of uncompressed data that it can, has consumed every byte that it needed, has successfully reached the end of the deflate stream, and */ + /* if zlib headers and adler32 checking enabled that it has successfully checked the uncompressed data's adler32. If you call it again you'll just get TINFL_STATUS_DONE over and over again. */ + TINFL_STATUS_DONE = 0, + + /* This flag indicates the inflator MUST have more input data (even 1 byte) before it can make any more forward progress, or you need to clear the TINFL_FLAG_HAS_MORE_INPUT */ + /* flag on the next call if you don't have any more source data. If the source data was somehow corrupted it's also possible (but unlikely) for the inflator to keep on demanding input to */ + /* proceed, so be sure to properly set the TINFL_FLAG_HAS_MORE_INPUT flag. */ + TINFL_STATUS_NEEDS_MORE_INPUT = 1, + + /* This flag indicates the inflator definitely has 1 or more bytes of uncompressed data available, but it cannot write this data into the output buffer. */ + /* Note if the source compressed data was corrupted it's possible for the inflator to return a lot of uncompressed data to the caller. I've been assuming you know how much uncompressed data to expect */ + /* (either exact or worst case) and will stop calling the inflator and fail after receiving too much. In pure streaming scenarios where you have no idea how many bytes to expect this may not be possible */ + /* so I may need to add some code to address this. */ + TINFL_STATUS_HAS_MORE_OUTPUT = 2 + } tinfl_status; + +/* Initializes the decompressor to its initial state. */ +#define tinfl_init(r) \ + do \ + { \ + (r)->m_state = 0; \ + } \ + MZ_MACRO_END +#define tinfl_get_adler32(r) (r)->m_check_adler32 + + /* Main low-level decompressor coroutine function. This is the only function actually needed for decompression. All the other functions are just high-level helpers for improved usability. */ + /* This is a universal API, i.e. it can be used as a building block to build any desired higher level decompression API. In the limit case, it can be called once per every byte input or output. */ + MINIZ_EXPORT tinfl_status tinfl_decompress(tinfl_decompressor *r, const mz_uint8 *pIn_buf_next, size_t *pIn_buf_size, mz_uint8 *pOut_buf_start, mz_uint8 *pOut_buf_next, size_t *pOut_buf_size, const mz_uint32 decomp_flags); + + /* Internal/private bits follow. */ + enum + { + TINFL_MAX_HUFF_TABLES = 3, + TINFL_MAX_HUFF_SYMBOLS_0 = 288, + TINFL_MAX_HUFF_SYMBOLS_1 = 32, + TINFL_MAX_HUFF_SYMBOLS_2 = 19, + TINFL_FAST_LOOKUP_BITS = 10, + TINFL_FAST_LOOKUP_SIZE = 1 << TINFL_FAST_LOOKUP_BITS + }; + +#if MINIZ_HAS_64BIT_REGISTERS +#define TINFL_USE_64BIT_BITBUF 1 +#else +#define TINFL_USE_64BIT_BITBUF 0 +#endif + +#if TINFL_USE_64BIT_BITBUF + typedef mz_uint64 tinfl_bit_buf_t; +#define TINFL_BITBUF_SIZE (64) +#else +typedef mz_uint32 tinfl_bit_buf_t; +#define TINFL_BITBUF_SIZE (32) +#endif + + struct tinfl_decompressor_tag + { + mz_uint32 m_state, m_num_bits, m_zhdr0, m_zhdr1, m_z_adler32, m_final, m_type, m_check_adler32, m_dist, m_counter, m_num_extra, m_table_sizes[TINFL_MAX_HUFF_TABLES]; + tinfl_bit_buf_t m_bit_buf; + size_t m_dist_from_out_buf_start; + mz_int16 m_look_up[TINFL_MAX_HUFF_TABLES][TINFL_FAST_LOOKUP_SIZE]; + mz_int16 m_tree_0[TINFL_MAX_HUFF_SYMBOLS_0 * 2]; + mz_int16 m_tree_1[TINFL_MAX_HUFF_SYMBOLS_1 * 2]; + mz_int16 m_tree_2[TINFL_MAX_HUFF_SYMBOLS_2 * 2]; + mz_uint8 m_code_size_0[TINFL_MAX_HUFF_SYMBOLS_0]; + mz_uint8 m_code_size_1[TINFL_MAX_HUFF_SYMBOLS_1]; + mz_uint8 m_code_size_2[TINFL_MAX_HUFF_SYMBOLS_2]; + mz_uint8 m_raw_header[4], m_len_codes[TINFL_MAX_HUFF_SYMBOLS_0 + TINFL_MAX_HUFF_SYMBOLS_1 + 137]; + }; + +#ifdef __cplusplus +} +#endif + +#endif /*#ifndef MINIZ_NO_INFLATE_APIS*/ diff --git a/src/helpers/miniz/miniz_zip.c b/src/helpers/miniz/miniz_zip.c new file mode 100644 index 000000000..303c732ce --- /dev/null +++ b/src/helpers/miniz/miniz_zip.c @@ -0,0 +1,4895 @@ +/************************************************************************** + * + * Copyright 2013-2014 RAD Game Tools and Valve Software + * Copyright 2010-2014 Rich Geldreich and Tenacious Software LLC + * Copyright 2016 Martin Raiber + * All Rights Reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + * + **************************************************************************/ +#include "miniz.h" + +#ifndef MINIZ_NO_ARCHIVE_APIS + +#ifdef __cplusplus +extern "C" +{ +#endif + + /* ------------------- .ZIP archive reading */ + +#ifdef MINIZ_NO_STDIO +#define MZ_FILE void * +#else +#include + +#if defined(_MSC_VER) || defined(__MINGW64__) || defined(__MINGW32__) + +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef __cplusplus +#define MICROSOFT_WINDOWS_WINBASE_H_DEFINE_INTERLOCKED_CPLUSPLUS_OVERLOADS 0 +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include + +static WCHAR *mz_utf8z_to_widechar(const char *str) +{ + int reqChars = MultiByteToWideChar(CP_UTF8, 0, str, -1, NULL, 0); + WCHAR *wStr = (WCHAR *)malloc(reqChars * sizeof(WCHAR)); + MultiByteToWideChar(CP_UTF8, 0, str, -1, wStr, reqChars); + return wStr; +} + +static FILE *mz_fopen(const char *pFilename, const char *pMode) +{ + WCHAR *wFilename = mz_utf8z_to_widechar(pFilename); + WCHAR *wMode = mz_utf8z_to_widechar(pMode); + FILE *pFile = NULL; + errno_t err = _wfopen_s(&pFile, wFilename, wMode); + free(wFilename); + free(wMode); + return err ? NULL : pFile; +} + +static FILE *mz_freopen(const char *pPath, const char *pMode, FILE *pStream) +{ + WCHAR *wPath = mz_utf8z_to_widechar(pPath); + WCHAR *wMode = mz_utf8z_to_widechar(pMode); + FILE *pFile = NULL; + errno_t err = _wfreopen_s(&pFile, wPath, wMode, pStream); + free(wPath); + free(wMode); + return err ? NULL : pFile; +} + +#if defined(__MINGW32__) +static int mz_stat(const char *path, struct _stat *buffer) +{ + WCHAR *wPath = mz_utf8z_to_widechar(path); + int res = _wstat(wPath, buffer); + free(wPath); + return res; +} +#else +static int mz_stat64(const char *path, struct __stat64 *buffer) +{ + WCHAR *wPath = mz_utf8z_to_widechar(path); + int res = _wstat64(wPath, buffer); + free(wPath); + return res; +} +#endif + +#ifndef MINIZ_NO_TIME +#include +#endif +#define MZ_FOPEN mz_fopen +#define MZ_FCLOSE fclose +#define MZ_FREAD fread +#define MZ_FWRITE fwrite +#define MZ_FTELL64 _ftelli64 +#define MZ_FSEEK64 _fseeki64 +#if defined(__MINGW32__) +#define MZ_FILE_STAT_STRUCT _stat +#define MZ_FILE_STAT mz_stat +#else +#define MZ_FILE_STAT_STRUCT _stat64 +#define MZ_FILE_STAT mz_stat64 +#endif +#define MZ_FFLUSH fflush +#define MZ_FREOPEN mz_freopen +#define MZ_DELETE_FILE remove + +#elif defined(__WATCOMC__) +#ifndef MINIZ_NO_TIME +#include +#endif +#define MZ_FOPEN(f, m) fopen(f, m) +#define MZ_FCLOSE fclose +#define MZ_FREAD fread +#define MZ_FWRITE fwrite +#define MZ_FTELL64 _ftelli64 +#define MZ_FSEEK64 _fseeki64 +#define MZ_FILE_STAT_STRUCT stat +#define MZ_FILE_STAT stat +#define MZ_FFLUSH fflush +#define MZ_FREOPEN(f, m, s) freopen(f, m, s) +#define MZ_DELETE_FILE remove + +#elif defined(__TINYC__) +#ifndef MINIZ_NO_TIME +#include +#endif +#define MZ_FOPEN(f, m) fopen(f, m) +#define MZ_FCLOSE fclose +#define MZ_FREAD fread +#define MZ_FWRITE fwrite +#define MZ_FTELL64 ftell +#define MZ_FSEEK64 fseek +#define MZ_FILE_STAT_STRUCT stat +#define MZ_FILE_STAT stat +#define MZ_FFLUSH fflush +#define MZ_FREOPEN(f, m, s) freopen(f, m, s) +#define MZ_DELETE_FILE remove + +#elif defined(__USE_LARGEFILE64) /* gcc, clang */ +#ifndef MINIZ_NO_TIME +#include +#endif +#define MZ_FOPEN(f, m) fopen64(f, m) +#define MZ_FCLOSE fclose +#define MZ_FREAD fread +#define MZ_FWRITE fwrite +#define MZ_FTELL64 ftello64 +#define MZ_FSEEK64 fseeko64 +#define MZ_FILE_STAT_STRUCT stat64 +#define MZ_FILE_STAT stat64 +#define MZ_FFLUSH fflush +#define MZ_FREOPEN(p, m, s) freopen64(p, m, s) +#define MZ_DELETE_FILE remove + +#elif defined(__APPLE__) || defined(__FreeBSD__) || (defined(__linux__) && defined(__x86_64__)) +#ifndef MINIZ_NO_TIME +#include +#endif +#define MZ_FOPEN(f, m) fopen(f, m) +#define MZ_FCLOSE fclose +#define MZ_FREAD fread +#define MZ_FWRITE fwrite +#define MZ_FTELL64 ftello +#define MZ_FSEEK64 fseeko +#define MZ_FILE_STAT_STRUCT stat +#define MZ_FILE_STAT stat +#define MZ_FFLUSH fflush +#define MZ_FREOPEN(p, m, s) freopen(p, m, s) +#define MZ_DELETE_FILE remove + +#else +#pragma message("Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.") +#ifndef MINIZ_NO_TIME +#include +#endif +#define MZ_FOPEN(f, m) fopen(f, m) +#define MZ_FCLOSE fclose +#define MZ_FREAD fread +#define MZ_FWRITE fwrite +#ifdef __STRICT_ANSI__ +#define MZ_FTELL64 ftell +#define MZ_FSEEK64 fseek +#else +#define MZ_FTELL64 ftello +#define MZ_FSEEK64 fseeko +#endif +#define MZ_FILE_STAT_STRUCT stat +#define MZ_FILE_STAT stat +#define MZ_FFLUSH fflush +#define MZ_FREOPEN(f, m, s) freopen(f, m, s) +#define MZ_DELETE_FILE remove +#endif /* #ifdef _MSC_VER */ +#endif /* #ifdef MINIZ_NO_STDIO */ + +#define MZ_TOLOWER(c) ((((c) >= 'A') && ((c) <= 'Z')) ? ((c) - 'A' + 'a') : (c)) + + /* Various ZIP archive enums. To completely avoid cross platform compiler alignment and platform endian issues, miniz.c doesn't use structs for any of this stuff. */ + enum + { + /* ZIP archive identifiers and record sizes */ + MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIG = 0x06054b50, + MZ_ZIP_CENTRAL_DIR_HEADER_SIG = 0x02014b50, + MZ_ZIP_LOCAL_DIR_HEADER_SIG = 0x04034b50, + MZ_ZIP_LOCAL_DIR_HEADER_SIZE = 30, + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE = 46, + MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE = 22, + + /* ZIP64 archive identifier and record sizes */ + MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIG = 0x06064b50, + MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIG = 0x07064b50, + MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE = 56, + MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE = 20, + MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID = 0x0001, + MZ_ZIP_DATA_DESCRIPTOR_ID = 0x08074b50, + MZ_ZIP_DATA_DESCRIPTER_SIZE64 = 24, + MZ_ZIP_DATA_DESCRIPTER_SIZE32 = 16, + + /* Central directory header record offsets */ + MZ_ZIP_CDH_SIG_OFS = 0, + MZ_ZIP_CDH_VERSION_MADE_BY_OFS = 4, + MZ_ZIP_CDH_VERSION_NEEDED_OFS = 6, + MZ_ZIP_CDH_BIT_FLAG_OFS = 8, + MZ_ZIP_CDH_METHOD_OFS = 10, + MZ_ZIP_CDH_FILE_TIME_OFS = 12, + MZ_ZIP_CDH_FILE_DATE_OFS = 14, + MZ_ZIP_CDH_CRC32_OFS = 16, + MZ_ZIP_CDH_COMPRESSED_SIZE_OFS = 20, + MZ_ZIP_CDH_DECOMPRESSED_SIZE_OFS = 24, + MZ_ZIP_CDH_FILENAME_LEN_OFS = 28, + MZ_ZIP_CDH_EXTRA_LEN_OFS = 30, + MZ_ZIP_CDH_COMMENT_LEN_OFS = 32, + MZ_ZIP_CDH_DISK_START_OFS = 34, + MZ_ZIP_CDH_INTERNAL_ATTR_OFS = 36, + MZ_ZIP_CDH_EXTERNAL_ATTR_OFS = 38, + MZ_ZIP_CDH_LOCAL_HEADER_OFS = 42, + + /* Local directory header offsets */ + MZ_ZIP_LDH_SIG_OFS = 0, + MZ_ZIP_LDH_VERSION_NEEDED_OFS = 4, + MZ_ZIP_LDH_BIT_FLAG_OFS = 6, + MZ_ZIP_LDH_METHOD_OFS = 8, + MZ_ZIP_LDH_FILE_TIME_OFS = 10, + MZ_ZIP_LDH_FILE_DATE_OFS = 12, + MZ_ZIP_LDH_CRC32_OFS = 14, + MZ_ZIP_LDH_COMPRESSED_SIZE_OFS = 18, + MZ_ZIP_LDH_DECOMPRESSED_SIZE_OFS = 22, + MZ_ZIP_LDH_FILENAME_LEN_OFS = 26, + MZ_ZIP_LDH_EXTRA_LEN_OFS = 28, + MZ_ZIP_LDH_BIT_FLAG_HAS_LOCATOR = 1 << 3, + + /* End of central directory offsets */ + MZ_ZIP_ECDH_SIG_OFS = 0, + MZ_ZIP_ECDH_NUM_THIS_DISK_OFS = 4, + MZ_ZIP_ECDH_NUM_DISK_CDIR_OFS = 6, + MZ_ZIP_ECDH_CDIR_NUM_ENTRIES_ON_DISK_OFS = 8, + MZ_ZIP_ECDH_CDIR_TOTAL_ENTRIES_OFS = 10, + MZ_ZIP_ECDH_CDIR_SIZE_OFS = 12, + MZ_ZIP_ECDH_CDIR_OFS_OFS = 16, + MZ_ZIP_ECDH_COMMENT_SIZE_OFS = 20, + + /* ZIP64 End of central directory locator offsets */ + MZ_ZIP64_ECDL_SIG_OFS = 0, /* 4 bytes */ + MZ_ZIP64_ECDL_NUM_DISK_CDIR_OFS = 4, /* 4 bytes */ + MZ_ZIP64_ECDL_REL_OFS_TO_ZIP64_ECDR_OFS = 8, /* 8 bytes */ + MZ_ZIP64_ECDL_TOTAL_NUMBER_OF_DISKS_OFS = 16, /* 4 bytes */ + + /* ZIP64 End of central directory header offsets */ + MZ_ZIP64_ECDH_SIG_OFS = 0, /* 4 bytes */ + MZ_ZIP64_ECDH_SIZE_OF_RECORD_OFS = 4, /* 8 bytes */ + MZ_ZIP64_ECDH_VERSION_MADE_BY_OFS = 12, /* 2 bytes */ + MZ_ZIP64_ECDH_VERSION_NEEDED_OFS = 14, /* 2 bytes */ + MZ_ZIP64_ECDH_NUM_THIS_DISK_OFS = 16, /* 4 bytes */ + MZ_ZIP64_ECDH_NUM_DISK_CDIR_OFS = 20, /* 4 bytes */ + MZ_ZIP64_ECDH_CDIR_NUM_ENTRIES_ON_DISK_OFS = 24, /* 8 bytes */ + MZ_ZIP64_ECDH_CDIR_TOTAL_ENTRIES_OFS = 32, /* 8 bytes */ + MZ_ZIP64_ECDH_CDIR_SIZE_OFS = 40, /* 8 bytes */ + MZ_ZIP64_ECDH_CDIR_OFS_OFS = 48, /* 8 bytes */ + MZ_ZIP_VERSION_MADE_BY_DOS_FILESYSTEM_ID = 0, + MZ_ZIP_DOS_DIR_ATTRIBUTE_BITFLAG = 0x10, + MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_IS_ENCRYPTED = 1, + MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_COMPRESSED_PATCH_FLAG = 32, + MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_USES_STRONG_ENCRYPTION = 64, + MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_LOCAL_DIR_IS_MASKED = 8192, + MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_UTF8 = 1 << 11 + }; + + typedef struct + { + void *m_p; + size_t m_size, m_capacity; + mz_uint m_element_size; + } mz_zip_array; + + struct mz_zip_internal_state_tag + { + mz_zip_array m_central_dir; + mz_zip_array m_central_dir_offsets; + mz_zip_array m_sorted_central_dir_offsets; + + /* The flags passed in when the archive is initially opened. */ + mz_uint32 m_init_flags; + + /* MZ_TRUE if the archive has a zip64 end of central directory headers, etc. */ + mz_bool m_zip64; + + /* MZ_TRUE if we found zip64 extended info in the central directory (m_zip64 will also be slammed to true too, even if we didn't find a zip64 end of central dir header, etc.) */ + mz_bool m_zip64_has_extended_info_fields; + + /* These fields are used by the file, FILE, memory, and memory/heap read/write helpers. */ + MZ_FILE *m_pFile; + mz_uint64 m_file_archive_start_ofs; + + void *m_pMem; + size_t m_mem_size; + size_t m_mem_capacity; + }; + +#define MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(array_ptr, element_size) (array_ptr)->m_element_size = element_size + +#if defined(DEBUG) || defined(_DEBUG) + static MZ_FORCEINLINE mz_uint mz_zip_array_range_check(const mz_zip_array *pArray, mz_uint index) + { + MZ_ASSERT(index < pArray->m_size); + return index; + } +#define MZ_ZIP_ARRAY_ELEMENT(array_ptr, element_type, index) ((element_type *)((array_ptr)->m_p))[mz_zip_array_range_check(array_ptr, index)] +#else +#define MZ_ZIP_ARRAY_ELEMENT(array_ptr, element_type, index) ((element_type *)((array_ptr)->m_p))[index] +#endif + + static MZ_FORCEINLINE void mz_zip_array_init(mz_zip_array *pArray, mz_uint32 element_size) + { + memset(pArray, 0, sizeof(mz_zip_array)); + pArray->m_element_size = element_size; + } + + static MZ_FORCEINLINE void mz_zip_array_clear(mz_zip_archive *pZip, mz_zip_array *pArray) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pArray->m_p); + memset(pArray, 0, sizeof(mz_zip_array)); + } + + static mz_bool mz_zip_array_ensure_capacity(mz_zip_archive *pZip, mz_zip_array *pArray, size_t min_new_capacity, mz_uint growing) + { + void *pNew_p; + size_t new_capacity = min_new_capacity; + MZ_ASSERT(pArray->m_element_size); + if (pArray->m_capacity >= min_new_capacity) + return MZ_TRUE; + if (growing) + { + new_capacity = MZ_MAX(1, pArray->m_capacity); + while (new_capacity < min_new_capacity) + new_capacity *= 2; + } + if (NULL == (pNew_p = pZip->m_pRealloc(pZip->m_pAlloc_opaque, pArray->m_p, pArray->m_element_size, new_capacity))) + return MZ_FALSE; + pArray->m_p = pNew_p; + pArray->m_capacity = new_capacity; + return MZ_TRUE; + } + + static MZ_FORCEINLINE mz_bool mz_zip_array_reserve(mz_zip_archive *pZip, mz_zip_array *pArray, size_t new_capacity, mz_uint growing) + { + if (new_capacity > pArray->m_capacity) + { + if (!mz_zip_array_ensure_capacity(pZip, pArray, new_capacity, growing)) + return MZ_FALSE; + } + return MZ_TRUE; + } + + static MZ_FORCEINLINE mz_bool mz_zip_array_resize(mz_zip_archive *pZip, mz_zip_array *pArray, size_t new_size, mz_uint growing) + { + if (new_size > pArray->m_capacity) + { + if (!mz_zip_array_ensure_capacity(pZip, pArray, new_size, growing)) + return MZ_FALSE; + } + pArray->m_size = new_size; + return MZ_TRUE; + } + + static MZ_FORCEINLINE mz_bool mz_zip_array_ensure_room(mz_zip_archive *pZip, mz_zip_array *pArray, size_t n) + { + return mz_zip_array_reserve(pZip, pArray, pArray->m_size + n, MZ_TRUE); + } + + static MZ_FORCEINLINE mz_bool mz_zip_array_push_back(mz_zip_archive *pZip, mz_zip_array *pArray, const void *pElements, size_t n) + { + size_t orig_size = pArray->m_size; + if (!mz_zip_array_resize(pZip, pArray, orig_size + n, MZ_TRUE)) + return MZ_FALSE; + if (n > 0) + memcpy((mz_uint8 *)pArray->m_p + orig_size * pArray->m_element_size, pElements, n * pArray->m_element_size); + return MZ_TRUE; + } + +#ifndef MINIZ_NO_TIME + static MZ_TIME_T mz_zip_dos_to_time_t(int dos_time, int dos_date) + { + struct tm tm; + memset(&tm, 0, sizeof(tm)); + tm.tm_isdst = -1; + tm.tm_year = ((dos_date >> 9) & 127) + 1980 - 1900; + tm.tm_mon = ((dos_date >> 5) & 15) - 1; + tm.tm_mday = dos_date & 31; + tm.tm_hour = (dos_time >> 11) & 31; + tm.tm_min = (dos_time >> 5) & 63; + tm.tm_sec = (dos_time << 1) & 62; + return mktime(&tm); + } + +#ifndef MINIZ_NO_ARCHIVE_WRITING_APIS + static void mz_zip_time_t_to_dos_time(MZ_TIME_T time, mz_uint16 *pDOS_time, mz_uint16 *pDOS_date) + { +#ifdef _MSC_VER + struct tm tm_struct; + struct tm *tm = &tm_struct; + errno_t err = localtime_s(tm, &time); + if (err) + { + *pDOS_date = 0; + *pDOS_time = 0; + return; + } +#else + struct tm *tm = localtime(&time); +#endif /* #ifdef _MSC_VER */ + + *pDOS_time = (mz_uint16)(((tm->tm_hour) << 11) + ((tm->tm_min) << 5) + ((tm->tm_sec) >> 1)); + *pDOS_date = (mz_uint16)(((tm->tm_year + 1900 - 1980) << 9) + ((tm->tm_mon + 1) << 5) + tm->tm_mday); + } +#endif /* MINIZ_NO_ARCHIVE_WRITING_APIS */ + +#ifndef MINIZ_NO_STDIO +#ifndef MINIZ_NO_ARCHIVE_WRITING_APIS + static mz_bool mz_zip_get_file_modified_time(const char *pFilename, MZ_TIME_T *pTime) + { + struct MZ_FILE_STAT_STRUCT file_stat; + + /* On Linux with x86 glibc, this call will fail on large files (I think >= 0x80000000 bytes) unless you compiled with _LARGEFILE64_SOURCE. Argh. */ + if (MZ_FILE_STAT(pFilename, &file_stat) != 0) + return MZ_FALSE; + + *pTime = file_stat.st_mtime; + + return MZ_TRUE; + } +#endif /* #ifndef MINIZ_NO_ARCHIVE_WRITING_APIS*/ + + static mz_bool mz_zip_set_file_times(const char *pFilename, MZ_TIME_T access_time, MZ_TIME_T modified_time) + { + struct utimbuf t; + + memset(&t, 0, sizeof(t)); + t.actime = access_time; + t.modtime = modified_time; + + return !utime(pFilename, &t); + } +#endif /* #ifndef MINIZ_NO_STDIO */ +#endif /* #ifndef MINIZ_NO_TIME */ + + static MZ_FORCEINLINE mz_bool mz_zip_set_error(mz_zip_archive *pZip, mz_zip_error err_num) + { + if (pZip) + pZip->m_last_error = err_num; + return MZ_FALSE; + } + + static mz_bool mz_zip_reader_init_internal(mz_zip_archive *pZip, mz_uint flags) + { + (void)flags; + if ((!pZip) || (pZip->m_pState) || (pZip->m_zip_mode != MZ_ZIP_MODE_INVALID)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (!pZip->m_pAlloc) + pZip->m_pAlloc = miniz_def_alloc_func; + if (!pZip->m_pFree) + pZip->m_pFree = miniz_def_free_func; + if (!pZip->m_pRealloc) + pZip->m_pRealloc = miniz_def_realloc_func; + + pZip->m_archive_size = 0; + pZip->m_central_directory_file_ofs = 0; + pZip->m_total_files = 0; + pZip->m_last_error = MZ_ZIP_NO_ERROR; + + if (NULL == (pZip->m_pState = (mz_zip_internal_state *)pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, sizeof(mz_zip_internal_state)))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + memset(pZip->m_pState, 0, sizeof(mz_zip_internal_state)); + MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(&pZip->m_pState->m_central_dir, sizeof(mz_uint8)); + MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(&pZip->m_pState->m_central_dir_offsets, sizeof(mz_uint32)); + MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(&pZip->m_pState->m_sorted_central_dir_offsets, sizeof(mz_uint32)); + pZip->m_pState->m_init_flags = flags; + pZip->m_pState->m_zip64 = MZ_FALSE; + pZip->m_pState->m_zip64_has_extended_info_fields = MZ_FALSE; + + pZip->m_zip_mode = MZ_ZIP_MODE_READING; + + return MZ_TRUE; + } + + static MZ_FORCEINLINE mz_bool mz_zip_reader_filename_less(const mz_zip_array *pCentral_dir_array, const mz_zip_array *pCentral_dir_offsets, mz_uint l_index, mz_uint r_index) + { + const mz_uint8 *pL = &MZ_ZIP_ARRAY_ELEMENT(pCentral_dir_array, mz_uint8, MZ_ZIP_ARRAY_ELEMENT(pCentral_dir_offsets, mz_uint32, l_index)), *pE; + const mz_uint8 *pR = &MZ_ZIP_ARRAY_ELEMENT(pCentral_dir_array, mz_uint8, MZ_ZIP_ARRAY_ELEMENT(pCentral_dir_offsets, mz_uint32, r_index)); + mz_uint l_len = MZ_READ_LE16(pL + MZ_ZIP_CDH_FILENAME_LEN_OFS), r_len = MZ_READ_LE16(pR + MZ_ZIP_CDH_FILENAME_LEN_OFS); + mz_uint8 l = 0, r = 0; + pL += MZ_ZIP_CENTRAL_DIR_HEADER_SIZE; + pR += MZ_ZIP_CENTRAL_DIR_HEADER_SIZE; + pE = pL + MZ_MIN(l_len, r_len); + while (pL < pE) + { + if ((l = MZ_TOLOWER(*pL)) != (r = MZ_TOLOWER(*pR))) + break; + pL++; + pR++; + } + return (pL == pE) ? (l_len < r_len) : (l < r); + } + +#define MZ_SWAP_UINT32(a, b) \ + do \ + { \ + mz_uint32 t = a; \ + a = b; \ + b = t; \ + } \ + MZ_MACRO_END + + /* Heap sort of lowercased filenames, used to help accelerate plain central directory searches by mz_zip_reader_locate_file(). (Could also use qsort(), but it could allocate memory.) */ + static void mz_zip_reader_sort_central_dir_offsets_by_filename(mz_zip_archive *pZip) + { + mz_zip_internal_state *pState = pZip->m_pState; + const mz_zip_array *pCentral_dir_offsets = &pState->m_central_dir_offsets; + const mz_zip_array *pCentral_dir = &pState->m_central_dir; + mz_uint32 *pIndices; + mz_uint32 start, end; + const mz_uint32 size = pZip->m_total_files; + + if (size <= 1U) + return; + + pIndices = &MZ_ZIP_ARRAY_ELEMENT(&pState->m_sorted_central_dir_offsets, mz_uint32, 0); + + start = (size - 2U) >> 1U; + for (;;) + { + mz_uint64 child, root = start; + for (;;) + { + if ((child = (root << 1U) + 1U) >= size) + break; + child += (((child + 1U) < size) && (mz_zip_reader_filename_less(pCentral_dir, pCentral_dir_offsets, pIndices[child], pIndices[child + 1U]))); + if (!mz_zip_reader_filename_less(pCentral_dir, pCentral_dir_offsets, pIndices[root], pIndices[child])) + break; + MZ_SWAP_UINT32(pIndices[root], pIndices[child]); + root = child; + } + if (!start) + break; + start--; + } + + end = size - 1; + while (end > 0) + { + mz_uint64 child, root = 0; + MZ_SWAP_UINT32(pIndices[end], pIndices[0]); + for (;;) + { + if ((child = (root << 1U) + 1U) >= end) + break; + child += (((child + 1U) < end) && mz_zip_reader_filename_less(pCentral_dir, pCentral_dir_offsets, pIndices[child], pIndices[child + 1U])); + if (!mz_zip_reader_filename_less(pCentral_dir, pCentral_dir_offsets, pIndices[root], pIndices[child])) + break; + MZ_SWAP_UINT32(pIndices[root], pIndices[child]); + root = child; + } + end--; + } + } + + static mz_bool mz_zip_reader_locate_header_sig(mz_zip_archive *pZip, mz_uint32 record_sig, mz_uint32 record_size, mz_int64 *pOfs) + { + mz_int64 cur_file_ofs; + mz_uint32 buf_u32[4096 / sizeof(mz_uint32)]; + mz_uint8 *pBuf = (mz_uint8 *)buf_u32; + + /* Basic sanity checks - reject files which are too small */ + if (pZip->m_archive_size < record_size) + return MZ_FALSE; + + /* Find the record by scanning the file from the end towards the beginning. */ + cur_file_ofs = MZ_MAX((mz_int64)pZip->m_archive_size - (mz_int64)sizeof(buf_u32), 0); + for (;;) + { + int i, n = (int)MZ_MIN(sizeof(buf_u32), pZip->m_archive_size - cur_file_ofs); + + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pBuf, n) != (mz_uint)n) + return MZ_FALSE; + + for (i = n - 4; i >= 0; --i) + { + mz_uint s = MZ_READ_LE32(pBuf + i); + if (s == record_sig) + { + if ((pZip->m_archive_size - (cur_file_ofs + i)) >= record_size) + break; + } + } + + if (i >= 0) + { + cur_file_ofs += i; + break; + } + + /* Give up if we've searched the entire file, or we've gone back "too far" (~64kb) */ + if ((!cur_file_ofs) || ((pZip->m_archive_size - cur_file_ofs) >= ((mz_uint64)(MZ_UINT16_MAX) + record_size))) + return MZ_FALSE; + + cur_file_ofs = MZ_MAX(cur_file_ofs - (sizeof(buf_u32) - 3), 0); + } + + *pOfs = cur_file_ofs; + return MZ_TRUE; + } + + static mz_bool mz_zip_reader_eocd64_valid(mz_zip_archive *pZip, uint64_t offset, uint8_t *buf) + { + if (pZip->m_pRead(pZip->m_pIO_opaque, offset, buf, MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE) == MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE) + { + if (MZ_READ_LE32(buf + MZ_ZIP64_ECDH_SIG_OFS) == MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIG) + { + return MZ_TRUE; + } + } + + return MZ_FALSE; + } + + static mz_bool mz_zip_reader_read_central_dir(mz_zip_archive *pZip, mz_uint flags) + { + mz_uint cdir_size = 0, cdir_entries_on_this_disk = 0, num_this_disk = 0, cdir_disk_index = 0; + mz_uint64 cdir_ofs = 0, eocd_ofs = 0, archive_ofs = 0; + mz_int64 cur_file_ofs = 0; + const mz_uint8 *p; + + mz_uint32 buf_u32[4096 / sizeof(mz_uint32)]; + mz_uint8 *pBuf = (mz_uint8 *)buf_u32; + mz_bool sort_central_dir = ((flags & MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY) == 0); + mz_uint32 zip64_end_of_central_dir_locator_u32[(MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pZip64_locator = (mz_uint8 *)zip64_end_of_central_dir_locator_u32; + + mz_uint32 zip64_end_of_central_dir_header_u32[(MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pZip64_end_of_central_dir = (mz_uint8 *)zip64_end_of_central_dir_header_u32; + + mz_uint64 zip64_end_of_central_dir_ofs = 0; + + /* Basic sanity checks - reject files which are too small, and check the first 4 bytes of the file to make sure a local header is there. */ + if (pZip->m_archive_size < MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + + if (!mz_zip_reader_locate_header_sig(pZip, MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIG, MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE, &cur_file_ofs)) + return mz_zip_set_error(pZip, MZ_ZIP_FAILED_FINDING_CENTRAL_DIR); + + eocd_ofs = cur_file_ofs; + /* Read and verify the end of central directory record. */ + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pBuf, MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) != MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + + if (MZ_READ_LE32(pBuf + MZ_ZIP_ECDH_SIG_OFS) != MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIG) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + + if (cur_file_ofs >= (MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE + MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE)) + { + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs - MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE, pZip64_locator, MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE) == MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE) + { + if (MZ_READ_LE32(pZip64_locator + MZ_ZIP64_ECDL_SIG_OFS) == MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIG) + { + pZip->m_pState->m_zip64 = MZ_TRUE; + } + } + } + + if (pZip->m_pState->m_zip64) + { + /* Try locating the EOCD64 right before the EOCD64 locator. This works even + * when the effective start of the zip header is not yet known. */ + if (cur_file_ofs < MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE + + MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + + zip64_end_of_central_dir_ofs = cur_file_ofs - + MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE - + MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE; + + if (!mz_zip_reader_eocd64_valid(pZip, zip64_end_of_central_dir_ofs, + pZip64_end_of_central_dir)) + { + /* That failed, try reading where the locator tells us to. */ + zip64_end_of_central_dir_ofs = MZ_READ_LE64( + pZip64_locator + MZ_ZIP64_ECDL_REL_OFS_TO_ZIP64_ECDR_OFS); + + if (zip64_end_of_central_dir_ofs > + (pZip->m_archive_size - MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE)) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + + if (!mz_zip_reader_eocd64_valid(pZip, zip64_end_of_central_dir_ofs, + pZip64_end_of_central_dir)) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + } + } + + pZip->m_total_files = MZ_READ_LE16(pBuf + MZ_ZIP_ECDH_CDIR_TOTAL_ENTRIES_OFS); + cdir_entries_on_this_disk = MZ_READ_LE16(pBuf + MZ_ZIP_ECDH_CDIR_NUM_ENTRIES_ON_DISK_OFS); + num_this_disk = MZ_READ_LE16(pBuf + MZ_ZIP_ECDH_NUM_THIS_DISK_OFS); + cdir_disk_index = MZ_READ_LE16(pBuf + MZ_ZIP_ECDH_NUM_DISK_CDIR_OFS); + cdir_size = MZ_READ_LE32(pBuf + MZ_ZIP_ECDH_CDIR_SIZE_OFS); + cdir_ofs = MZ_READ_LE32(pBuf + MZ_ZIP_ECDH_CDIR_OFS_OFS); + + if (pZip->m_pState->m_zip64) + { + mz_uint32 zip64_total_num_of_disks = MZ_READ_LE32(pZip64_locator + MZ_ZIP64_ECDL_TOTAL_NUMBER_OF_DISKS_OFS); + mz_uint64 zip64_cdir_total_entries = MZ_READ_LE64(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_CDIR_TOTAL_ENTRIES_OFS); + mz_uint64 zip64_cdir_total_entries_on_this_disk = MZ_READ_LE64(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_CDIR_NUM_ENTRIES_ON_DISK_OFS); + mz_uint64 zip64_size_of_end_of_central_dir_record = MZ_READ_LE64(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_SIZE_OF_RECORD_OFS); + mz_uint64 zip64_size_of_central_directory = MZ_READ_LE64(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_CDIR_SIZE_OFS); + + if (zip64_size_of_end_of_central_dir_record < (MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE - 12)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if (zip64_total_num_of_disks != 1U) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_MULTIDISK); + + /* Check for miniz's practical limits */ + if (zip64_cdir_total_entries > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + + pZip->m_total_files = (mz_uint32)zip64_cdir_total_entries; + + if (zip64_cdir_total_entries_on_this_disk > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + + cdir_entries_on_this_disk = (mz_uint32)zip64_cdir_total_entries_on_this_disk; + + /* Check for miniz's current practical limits (sorry, this should be enough for millions of files) */ + if (zip64_size_of_central_directory > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_CDIR_SIZE); + + cdir_size = (mz_uint32)zip64_size_of_central_directory; + + num_this_disk = MZ_READ_LE32(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_NUM_THIS_DISK_OFS); + + cdir_disk_index = MZ_READ_LE32(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_NUM_DISK_CDIR_OFS); + + cdir_ofs = MZ_READ_LE64(pZip64_end_of_central_dir + MZ_ZIP64_ECDH_CDIR_OFS_OFS); + } + + if (pZip->m_total_files != cdir_entries_on_this_disk) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_MULTIDISK); + + if (((num_this_disk | cdir_disk_index) != 0) && ((num_this_disk != 1) || (cdir_disk_index != 1))) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_MULTIDISK); + + if (cdir_size < (mz_uint64)pZip->m_total_files * MZ_ZIP_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if ((cdir_ofs + (mz_uint64)cdir_size) > pZip->m_archive_size) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if (eocd_ofs < cdir_ofs + cdir_size) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + /* The end of central dir follows the central dir, unless the zip file has + * some trailing data (e.g. it is appended to an executable file). */ + archive_ofs = eocd_ofs - (cdir_ofs + cdir_size); + if (pZip->m_pState->m_zip64) + { + if (archive_ofs < MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE + + MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + archive_ofs -= MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE + + MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE; + } + + /* Update the archive start position, but only if not specified. */ + if ((pZip->m_zip_type == MZ_ZIP_TYPE_FILE || pZip->m_zip_type == MZ_ZIP_TYPE_CFILE || + pZip->m_zip_type == MZ_ZIP_TYPE_USER) && pZip->m_pState->m_file_archive_start_ofs == 0) + { + pZip->m_pState->m_file_archive_start_ofs = archive_ofs; + pZip->m_archive_size -= archive_ofs; + } + + pZip->m_central_directory_file_ofs = cdir_ofs; + + if (pZip->m_total_files) + { + mz_uint i, n; + /* Read the entire central directory into a heap block, and allocate another heap block to hold the unsorted central dir file record offsets, and possibly another to hold the sorted indices. */ + if ((!mz_zip_array_resize(pZip, &pZip->m_pState->m_central_dir, cdir_size, MZ_FALSE)) || + (!mz_zip_array_resize(pZip, &pZip->m_pState->m_central_dir_offsets, pZip->m_total_files, MZ_FALSE))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + if (sort_central_dir) + { + if (!mz_zip_array_resize(pZip, &pZip->m_pState->m_sorted_central_dir_offsets, pZip->m_total_files, MZ_FALSE)) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (pZip->m_pRead(pZip->m_pIO_opaque, cdir_ofs, pZip->m_pState->m_central_dir.m_p, cdir_size) != cdir_size) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + + /* Now create an index into the central directory file records, do some basic sanity checking on each record */ + p = (const mz_uint8 *)pZip->m_pState->m_central_dir.m_p; + for (n = cdir_size, i = 0; i < pZip->m_total_files; ++i) + { + mz_uint total_header_size, disk_index, bit_flags, filename_size, ext_data_size; + mz_uint64 comp_size, decomp_size, local_header_ofs; + + if ((n < MZ_ZIP_CENTRAL_DIR_HEADER_SIZE) || (MZ_READ_LE32(p) != MZ_ZIP_CENTRAL_DIR_HEADER_SIG)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_central_dir_offsets, mz_uint32, i) = (mz_uint32)(p - (const mz_uint8 *)pZip->m_pState->m_central_dir.m_p); + + if (sort_central_dir) + MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_sorted_central_dir_offsets, mz_uint32, i) = i; + + comp_size = MZ_READ_LE32(p + MZ_ZIP_CDH_COMPRESSED_SIZE_OFS); + decomp_size = MZ_READ_LE32(p + MZ_ZIP_CDH_DECOMPRESSED_SIZE_OFS); + local_header_ofs = MZ_READ_LE32(p + MZ_ZIP_CDH_LOCAL_HEADER_OFS); + filename_size = MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS); + ext_data_size = MZ_READ_LE16(p + MZ_ZIP_CDH_EXTRA_LEN_OFS); + + if ((!pZip->m_pState->m_zip64_has_extended_info_fields) && + (ext_data_size) && + (MZ_MAX(MZ_MAX(comp_size, decomp_size), local_header_ofs) == MZ_UINT32_MAX)) + { + /* Attempt to find zip64 extended information field in the entry's extra data */ + mz_uint32 extra_size_remaining = ext_data_size; + + if (extra_size_remaining) + { + const mz_uint8 *pExtra_data; + void *buf = NULL; + + if (MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + filename_size + ext_data_size > n) + { + buf = MZ_MALLOC(ext_data_size); + if (buf == NULL) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + if (pZip->m_pRead(pZip->m_pIO_opaque, cdir_ofs + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + filename_size, buf, ext_data_size) != ext_data_size) + { + MZ_FREE(buf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + } + + pExtra_data = (mz_uint8 *)buf; + } + else + { + pExtra_data = p + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + filename_size; + } + + do + { + mz_uint32 field_id; + mz_uint32 field_data_size; + + if (extra_size_remaining < (sizeof(mz_uint16) * 2)) + { + MZ_FREE(buf); + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + field_id = MZ_READ_LE16(pExtra_data); + field_data_size = MZ_READ_LE16(pExtra_data + sizeof(mz_uint16)); + + if ((field_data_size + sizeof(mz_uint16) * 2) > extra_size_remaining) + { + MZ_FREE(buf); + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + if (field_id == MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID) + { + /* Ok, the archive didn't have any zip64 headers but it uses a zip64 extended information field so mark it as zip64 anyway (this can occur with infozip's zip util when it reads compresses files from stdin). */ + pZip->m_pState->m_zip64 = MZ_TRUE; + pZip->m_pState->m_zip64_has_extended_info_fields = MZ_TRUE; + break; + } + + pExtra_data += sizeof(mz_uint16) * 2 + field_data_size; + extra_size_remaining = extra_size_remaining - sizeof(mz_uint16) * 2 - field_data_size; + } while (extra_size_remaining); + + MZ_FREE(buf); + } + } + + /* I've seen archives that aren't marked as zip64 that uses zip64 ext data, argh */ + if ((comp_size != MZ_UINT32_MAX) && (decomp_size != MZ_UINT32_MAX)) + { + if (((!MZ_READ_LE32(p + MZ_ZIP_CDH_METHOD_OFS)) && (decomp_size != comp_size)) || (decomp_size && !comp_size)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + disk_index = MZ_READ_LE16(p + MZ_ZIP_CDH_DISK_START_OFS); + if ((disk_index == MZ_UINT16_MAX) || ((disk_index != num_this_disk) && (disk_index != 1))) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_MULTIDISK); + + if (comp_size != MZ_UINT32_MAX) + { + if (((mz_uint64)MZ_READ_LE32(p + MZ_ZIP_CDH_LOCAL_HEADER_OFS) + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + comp_size) > pZip->m_archive_size) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + bit_flags = MZ_READ_LE16(p + MZ_ZIP_CDH_BIT_FLAG_OFS); + if (bit_flags & MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_LOCAL_DIR_IS_MASKED) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_ENCRYPTION); + + if ((total_header_size = MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS) + MZ_READ_LE16(p + MZ_ZIP_CDH_EXTRA_LEN_OFS) + MZ_READ_LE16(p + MZ_ZIP_CDH_COMMENT_LEN_OFS)) > n) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + n -= total_header_size; + p += total_header_size; + } + } + + if (sort_central_dir) + mz_zip_reader_sort_central_dir_offsets_by_filename(pZip); + + return MZ_TRUE; + } + + void mz_zip_zero_struct(mz_zip_archive *pZip) + { + if (pZip) + MZ_CLEAR_PTR(pZip); + } + + static mz_bool mz_zip_reader_end_internal(mz_zip_archive *pZip, mz_bool set_last_error) + { + mz_bool status = MZ_TRUE; + + if (!pZip) + return MZ_FALSE; + + if ((!pZip->m_pState) || (!pZip->m_pAlloc) || (!pZip->m_pFree) || (pZip->m_zip_mode != MZ_ZIP_MODE_READING)) + { + if (set_last_error) + pZip->m_last_error = MZ_ZIP_INVALID_PARAMETER; + + return MZ_FALSE; + } + + if (pZip->m_pState) + { + mz_zip_internal_state *pState = pZip->m_pState; + pZip->m_pState = NULL; + + mz_zip_array_clear(pZip, &pState->m_central_dir); + mz_zip_array_clear(pZip, &pState->m_central_dir_offsets); + mz_zip_array_clear(pZip, &pState->m_sorted_central_dir_offsets); + +#ifndef MINIZ_NO_STDIO + if (pState->m_pFile) + { + if (pZip->m_zip_type == MZ_ZIP_TYPE_FILE) + { + if (MZ_FCLOSE(pState->m_pFile) == EOF) + { + if (set_last_error) + pZip->m_last_error = MZ_ZIP_FILE_CLOSE_FAILED; + status = MZ_FALSE; + } + } + pState->m_pFile = NULL; + } +#endif /* #ifndef MINIZ_NO_STDIO */ + + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + } + pZip->m_zip_mode = MZ_ZIP_MODE_INVALID; + + return status; + } + + mz_bool mz_zip_reader_end(mz_zip_archive *pZip) + { + return mz_zip_reader_end_internal(pZip, MZ_TRUE); + } + mz_bool mz_zip_reader_init(mz_zip_archive *pZip, mz_uint64 size, mz_uint flags) + { + if ((!pZip) || (!pZip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (!mz_zip_reader_init_internal(pZip, flags)) + return MZ_FALSE; + + pZip->m_zip_type = MZ_ZIP_TYPE_USER; + pZip->m_archive_size = size; + + if (!mz_zip_reader_read_central_dir(pZip, flags)) + { + mz_zip_reader_end_internal(pZip, MZ_FALSE); + return MZ_FALSE; + } + + return MZ_TRUE; + } + + static size_t mz_zip_mem_read_func(void *pOpaque, mz_uint64 file_ofs, void *pBuf, size_t n) + { + mz_zip_archive *pZip = (mz_zip_archive *)pOpaque; + size_t s = (file_ofs >= pZip->m_archive_size) ? 0 : (size_t)MZ_MIN(pZip->m_archive_size - file_ofs, n); + memcpy(pBuf, (const mz_uint8 *)pZip->m_pState->m_pMem + file_ofs, s); + return s; + } + + mz_bool mz_zip_reader_init_mem(mz_zip_archive *pZip, const void *pMem, size_t size, mz_uint flags) + { + if (!pMem) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (size < MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + + if (!mz_zip_reader_init_internal(pZip, flags)) + return MZ_FALSE; + + pZip->m_zip_type = MZ_ZIP_TYPE_MEMORY; + pZip->m_archive_size = size; + pZip->m_pRead = mz_zip_mem_read_func; + pZip->m_pIO_opaque = pZip; + pZip->m_pNeeds_keepalive = NULL; + +#ifdef __cplusplus + pZip->m_pState->m_pMem = const_cast(pMem); +#else + pZip->m_pState->m_pMem = (void *)pMem; +#endif + + pZip->m_pState->m_mem_size = size; + + if (!mz_zip_reader_read_central_dir(pZip, flags)) + { + mz_zip_reader_end_internal(pZip, MZ_FALSE); + return MZ_FALSE; + } + + return MZ_TRUE; + } + +#ifndef MINIZ_NO_STDIO + static size_t mz_zip_file_read_func(void *pOpaque, mz_uint64 file_ofs, void *pBuf, size_t n) + { + mz_zip_archive *pZip = (mz_zip_archive *)pOpaque; + mz_int64 cur_ofs = MZ_FTELL64(pZip->m_pState->m_pFile); + + file_ofs += pZip->m_pState->m_file_archive_start_ofs; + + if (((mz_int64)file_ofs < 0) || (((cur_ofs != (mz_int64)file_ofs)) && (MZ_FSEEK64(pZip->m_pState->m_pFile, (mz_int64)file_ofs, SEEK_SET)))) + return 0; + + return MZ_FREAD(pBuf, 1, n, pZip->m_pState->m_pFile); + } + + mz_bool mz_zip_reader_init_file(mz_zip_archive *pZip, const char *pFilename, mz_uint32 flags) + { + return mz_zip_reader_init_file_v2(pZip, pFilename, flags, 0, 0); + } + + mz_bool mz_zip_reader_init_file_v2(mz_zip_archive *pZip, const char *pFilename, mz_uint flags, mz_uint64 file_start_ofs, mz_uint64 archive_size) + { + mz_uint64 file_size; + MZ_FILE *pFile; + + if ((!pZip) || (!pFilename) || ((archive_size) && (archive_size < MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE))) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pFile = MZ_FOPEN(pFilename, (flags & MZ_ZIP_FLAG_READ_ALLOW_WRITING ) ? "r+b" : "rb"); + if (!pFile) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_OPEN_FAILED); + + file_size = archive_size; + if (!file_size) + { + if (MZ_FSEEK64(pFile, 0, SEEK_END)) + { + MZ_FCLOSE(pFile); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_SEEK_FAILED); + } + + file_size = MZ_FTELL64(pFile); + } + + /* TODO: Better sanity check archive_size and the # of actual remaining bytes */ + + if (file_size < MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) + { + MZ_FCLOSE(pFile); + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + } + + if (!mz_zip_reader_init_internal(pZip, flags)) + { + MZ_FCLOSE(pFile); + return MZ_FALSE; + } + + pZip->m_zip_type = MZ_ZIP_TYPE_FILE; + pZip->m_pRead = mz_zip_file_read_func; + pZip->m_pIO_opaque = pZip; + pZip->m_pState->m_pFile = pFile; + pZip->m_archive_size = file_size; + pZip->m_pState->m_file_archive_start_ofs = file_start_ofs; + + if (!mz_zip_reader_read_central_dir(pZip, flags)) + { + mz_zip_reader_end_internal(pZip, MZ_FALSE); + return MZ_FALSE; + } + + return MZ_TRUE; + } + + mz_bool mz_zip_reader_init_cfile(mz_zip_archive *pZip, MZ_FILE *pFile, mz_uint64 archive_size, mz_uint flags) + { + mz_uint64 cur_file_ofs; + + if ((!pZip) || (!pFile)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_OPEN_FAILED); + + cur_file_ofs = MZ_FTELL64(pFile); + + if (!archive_size) + { + if (MZ_FSEEK64(pFile, 0, SEEK_END)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_SEEK_FAILED); + + archive_size = MZ_FTELL64(pFile) - cur_file_ofs; + + if (archive_size < MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_NOT_AN_ARCHIVE); + } + + if (!mz_zip_reader_init_internal(pZip, flags)) + return MZ_FALSE; + + pZip->m_zip_type = MZ_ZIP_TYPE_CFILE; + pZip->m_pRead = mz_zip_file_read_func; + + pZip->m_pIO_opaque = pZip; + pZip->m_pState->m_pFile = pFile; + pZip->m_archive_size = archive_size; + pZip->m_pState->m_file_archive_start_ofs = cur_file_ofs; + + if (!mz_zip_reader_read_central_dir(pZip, flags)) + { + mz_zip_reader_end_internal(pZip, MZ_FALSE); + return MZ_FALSE; + } + + return MZ_TRUE; + } + +#endif /* #ifndef MINIZ_NO_STDIO */ + + static MZ_FORCEINLINE const mz_uint8 *mz_zip_get_cdh(mz_zip_archive *pZip, mz_uint file_index) + { + if ((!pZip) || (!pZip->m_pState) || (file_index >= pZip->m_total_files)) + return NULL; + return &MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_central_dir, mz_uint8, MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_central_dir_offsets, mz_uint32, file_index)); + } + + mz_bool mz_zip_reader_is_file_encrypted(mz_zip_archive *pZip, mz_uint file_index) + { + mz_uint m_bit_flag; + const mz_uint8 *p = mz_zip_get_cdh(pZip, file_index); + if (!p) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + return MZ_FALSE; + } + + m_bit_flag = MZ_READ_LE16(p + MZ_ZIP_CDH_BIT_FLAG_OFS); + return (m_bit_flag & (MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_IS_ENCRYPTED | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_USES_STRONG_ENCRYPTION)) != 0; + } + + mz_bool mz_zip_reader_is_file_supported(mz_zip_archive *pZip, mz_uint file_index) + { + mz_uint bit_flag; + mz_uint method; + + const mz_uint8 *p = mz_zip_get_cdh(pZip, file_index); + if (!p) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + return MZ_FALSE; + } + + method = MZ_READ_LE16(p + MZ_ZIP_CDH_METHOD_OFS); + bit_flag = MZ_READ_LE16(p + MZ_ZIP_CDH_BIT_FLAG_OFS); + + if ((method != 0) && (method != MZ_DEFLATED)) + { + mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_METHOD); + return MZ_FALSE; + } + + if (bit_flag & (MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_IS_ENCRYPTED | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_USES_STRONG_ENCRYPTION)) + { + mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_ENCRYPTION); + return MZ_FALSE; + } + + if (bit_flag & MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_COMPRESSED_PATCH_FLAG) + { + mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_FEATURE); + return MZ_FALSE; + } + + return MZ_TRUE; + } + + mz_bool mz_zip_reader_is_file_a_directory(mz_zip_archive *pZip, mz_uint file_index) + { + mz_uint filename_len, attribute_mapping_id, external_attr; + const mz_uint8 *p = mz_zip_get_cdh(pZip, file_index); + if (!p) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + return MZ_FALSE; + } + + filename_len = MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS); + if (filename_len) + { + if (*(p + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + filename_len - 1) == '/') + return MZ_TRUE; + } + + /* Bugfix: This code was also checking if the internal attribute was non-zero, which wasn't correct. */ + /* Most/all zip writers (hopefully) set DOS file/directory attributes in the low 16-bits, so check for the DOS directory flag and ignore the source OS ID in the created by field. */ + /* FIXME: Remove this check? Is it necessary - we already check the filename. */ + attribute_mapping_id = MZ_READ_LE16(p + MZ_ZIP_CDH_VERSION_MADE_BY_OFS) >> 8; + (void)attribute_mapping_id; + + external_attr = MZ_READ_LE32(p + MZ_ZIP_CDH_EXTERNAL_ATTR_OFS); + if ((external_attr & MZ_ZIP_DOS_DIR_ATTRIBUTE_BITFLAG) != 0) + { + return MZ_TRUE; + } + + return MZ_FALSE; + } + + static mz_bool mz_zip_file_stat_internal(mz_zip_archive *pZip, mz_uint file_index, const mz_uint8 *pCentral_dir_header, mz_zip_archive_file_stat *pStat, mz_bool *pFound_zip64_extra_data) + { + mz_uint n; + const mz_uint8 *p = pCentral_dir_header; + + if (pFound_zip64_extra_data) + *pFound_zip64_extra_data = MZ_FALSE; + + if ((!p) || (!pStat)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + /* Extract fields from the central directory record. */ + pStat->m_file_index = file_index; + pStat->m_central_dir_ofs = MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_central_dir_offsets, mz_uint32, file_index); + pStat->m_version_made_by = MZ_READ_LE16(p + MZ_ZIP_CDH_VERSION_MADE_BY_OFS); + pStat->m_version_needed = MZ_READ_LE16(p + MZ_ZIP_CDH_VERSION_NEEDED_OFS); + pStat->m_bit_flag = MZ_READ_LE16(p + MZ_ZIP_CDH_BIT_FLAG_OFS); + pStat->m_method = MZ_READ_LE16(p + MZ_ZIP_CDH_METHOD_OFS); +#ifndef MINIZ_NO_TIME + pStat->m_time = mz_zip_dos_to_time_t(MZ_READ_LE16(p + MZ_ZIP_CDH_FILE_TIME_OFS), MZ_READ_LE16(p + MZ_ZIP_CDH_FILE_DATE_OFS)); +#endif + pStat->m_crc32 = MZ_READ_LE32(p + MZ_ZIP_CDH_CRC32_OFS); + pStat->m_comp_size = MZ_READ_LE32(p + MZ_ZIP_CDH_COMPRESSED_SIZE_OFS); + pStat->m_uncomp_size = MZ_READ_LE32(p + MZ_ZIP_CDH_DECOMPRESSED_SIZE_OFS); + pStat->m_internal_attr = MZ_READ_LE16(p + MZ_ZIP_CDH_INTERNAL_ATTR_OFS); + pStat->m_external_attr = MZ_READ_LE32(p + MZ_ZIP_CDH_EXTERNAL_ATTR_OFS); + pStat->m_local_header_ofs = MZ_READ_LE32(p + MZ_ZIP_CDH_LOCAL_HEADER_OFS); + + /* Copy as much of the filename and comment as possible. */ + n = MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS); + n = MZ_MIN(n, MZ_ZIP_MAX_ARCHIVE_FILENAME_SIZE - 1); + memcpy(pStat->m_filename, p + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE, n); + pStat->m_filename[n] = '\0'; + + n = MZ_READ_LE16(p + MZ_ZIP_CDH_COMMENT_LEN_OFS); + n = MZ_MIN(n, MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE - 1); + pStat->m_comment_size = n; + memcpy(pStat->m_comment, p + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS) + MZ_READ_LE16(p + MZ_ZIP_CDH_EXTRA_LEN_OFS), n); + pStat->m_comment[n] = '\0'; + + /* Set some flags for convienance */ + pStat->m_is_directory = mz_zip_reader_is_file_a_directory(pZip, file_index); + pStat->m_is_encrypted = mz_zip_reader_is_file_encrypted(pZip, file_index); + pStat->m_is_supported = mz_zip_reader_is_file_supported(pZip, file_index); + + /* See if we need to read any zip64 extended information fields. */ + /* Confusingly, these zip64 fields can be present even on non-zip64 archives (Debian zip on a huge files from stdin piped to stdout creates them). */ + if (MZ_MAX(MZ_MAX(pStat->m_comp_size, pStat->m_uncomp_size), pStat->m_local_header_ofs) == MZ_UINT32_MAX) + { + /* Attempt to find zip64 extended information field in the entry's extra data */ + mz_uint32 extra_size_remaining = MZ_READ_LE16(p + MZ_ZIP_CDH_EXTRA_LEN_OFS); + + if (extra_size_remaining) + { + const mz_uint8 *pExtra_data = p + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS); + + do + { + mz_uint32 field_id; + mz_uint32 field_data_size; + + if (extra_size_remaining < (sizeof(mz_uint16) * 2)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + field_id = MZ_READ_LE16(pExtra_data); + field_data_size = MZ_READ_LE16(pExtra_data + sizeof(mz_uint16)); + + if ((field_data_size + sizeof(mz_uint16) * 2) > extra_size_remaining) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if (field_id == MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID) + { + const mz_uint8 *pField_data = pExtra_data + sizeof(mz_uint16) * 2; + mz_uint32 field_data_remaining = field_data_size; + + if (pFound_zip64_extra_data) + *pFound_zip64_extra_data = MZ_TRUE; + + if (pStat->m_uncomp_size == MZ_UINT32_MAX) + { + if (field_data_remaining < sizeof(mz_uint64)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + pStat->m_uncomp_size = MZ_READ_LE64(pField_data); + pField_data += sizeof(mz_uint64); + field_data_remaining -= sizeof(mz_uint64); + } + + if (pStat->m_comp_size == MZ_UINT32_MAX) + { + if (field_data_remaining < sizeof(mz_uint64)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + pStat->m_comp_size = MZ_READ_LE64(pField_data); + pField_data += sizeof(mz_uint64); + field_data_remaining -= sizeof(mz_uint64); + } + + if (pStat->m_local_header_ofs == MZ_UINT32_MAX) + { + if (field_data_remaining < sizeof(mz_uint64)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + pStat->m_local_header_ofs = MZ_READ_LE64(pField_data); + pField_data += sizeof(mz_uint64); + field_data_remaining -= sizeof(mz_uint64); + } + + break; + } + + pExtra_data += sizeof(mz_uint16) * 2 + field_data_size; + extra_size_remaining = extra_size_remaining - sizeof(mz_uint16) * 2 - field_data_size; + } while (extra_size_remaining); + } + } + + return MZ_TRUE; + } + + static MZ_FORCEINLINE mz_bool mz_zip_string_equal(const char *pA, const char *pB, mz_uint len, mz_uint flags) + { + mz_uint i; + if (flags & MZ_ZIP_FLAG_CASE_SENSITIVE) + return 0 == memcmp(pA, pB, len); + for (i = 0; i < len; ++i) + if (MZ_TOLOWER(pA[i]) != MZ_TOLOWER(pB[i])) + return MZ_FALSE; + return MZ_TRUE; + } + + static MZ_FORCEINLINE int mz_zip_filename_compare(const mz_zip_array *pCentral_dir_array, const mz_zip_array *pCentral_dir_offsets, mz_uint l_index, const char *pR, mz_uint r_len) + { + const mz_uint8 *pL = &MZ_ZIP_ARRAY_ELEMENT(pCentral_dir_array, mz_uint8, MZ_ZIP_ARRAY_ELEMENT(pCentral_dir_offsets, mz_uint32, l_index)), *pE; + mz_uint l_len = MZ_READ_LE16(pL + MZ_ZIP_CDH_FILENAME_LEN_OFS); + mz_uint8 l = 0, r = 0; + pL += MZ_ZIP_CENTRAL_DIR_HEADER_SIZE; + pE = pL + MZ_MIN(l_len, r_len); + while (pL < pE) + { + if ((l = MZ_TOLOWER(*pL)) != (r = MZ_TOLOWER(*pR))) + break; + pL++; + pR++; + } + return (pL == pE) ? (int)(l_len - r_len) : (l - r); + } + + static mz_bool mz_zip_locate_file_binary_search(mz_zip_archive *pZip, const char *pFilename, mz_uint32 *pIndex) + { + mz_zip_internal_state *pState = pZip->m_pState; + const mz_zip_array *pCentral_dir_offsets = &pState->m_central_dir_offsets; + const mz_zip_array *pCentral_dir = &pState->m_central_dir; + mz_uint32 *pIndices = &MZ_ZIP_ARRAY_ELEMENT(&pState->m_sorted_central_dir_offsets, mz_uint32, 0); + const mz_uint32 size = pZip->m_total_files; + const mz_uint filename_len = (mz_uint)strlen(pFilename); + + if (pIndex) + *pIndex = 0; + + if (size) + { + /* yes I could use uint32_t's, but then we would have to add some special case checks in the loop, argh, and */ + /* honestly the major expense here on 32-bit CPU's will still be the filename compare */ + mz_int64 l = 0, h = (mz_int64)size - 1; + + while (l <= h) + { + mz_int64 m = l + ((h - l) >> 1); + mz_uint32 file_index = pIndices[(mz_uint32)m]; + + int comp = mz_zip_filename_compare(pCentral_dir, pCentral_dir_offsets, file_index, pFilename, filename_len); + if (!comp) + { + if (pIndex) + *pIndex = file_index; + return MZ_TRUE; + } + else if (comp < 0) + l = m + 1; + else + h = m - 1; + } + } + + return mz_zip_set_error(pZip, MZ_ZIP_FILE_NOT_FOUND); + } + + int mz_zip_reader_locate_file(mz_zip_archive *pZip, const char *pName, const char *pComment, mz_uint flags) + { + mz_uint32 index; + if (!mz_zip_reader_locate_file_v2(pZip, pName, pComment, flags, &index)) + return -1; + else + return (int)index; + } + + mz_bool mz_zip_reader_locate_file_v2(mz_zip_archive *pZip, const char *pName, const char *pComment, mz_uint flags, mz_uint32 *pIndex) + { + mz_uint file_index; + size_t name_len, comment_len; + + if (pIndex) + *pIndex = 0; + + if ((!pZip) || (!pZip->m_pState) || (!pName)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + /* See if we can use a binary search */ + if (((pZip->m_pState->m_init_flags & MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY) == 0) && + (pZip->m_zip_mode == MZ_ZIP_MODE_READING) && + ((flags & (MZ_ZIP_FLAG_IGNORE_PATH | MZ_ZIP_FLAG_CASE_SENSITIVE)) == 0) && (!pComment) && (pZip->m_pState->m_sorted_central_dir_offsets.m_size)) + { + return mz_zip_locate_file_binary_search(pZip, pName, pIndex); + } + + /* Locate the entry by scanning the entire central directory */ + name_len = strlen(pName); + if (name_len > MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + comment_len = pComment ? strlen(pComment) : 0; + if (comment_len > MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + for (file_index = 0; file_index < pZip->m_total_files; file_index++) + { + const mz_uint8 *pHeader = &MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_central_dir, mz_uint8, MZ_ZIP_ARRAY_ELEMENT(&pZip->m_pState->m_central_dir_offsets, mz_uint32, file_index)); + mz_uint filename_len = MZ_READ_LE16(pHeader + MZ_ZIP_CDH_FILENAME_LEN_OFS); + const char *pFilename = (const char *)pHeader + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE; + if (filename_len < name_len) + continue; + if (comment_len) + { + mz_uint file_extra_len = MZ_READ_LE16(pHeader + MZ_ZIP_CDH_EXTRA_LEN_OFS), file_comment_len = MZ_READ_LE16(pHeader + MZ_ZIP_CDH_COMMENT_LEN_OFS); + const char *pFile_comment = pFilename + filename_len + file_extra_len; + if ((file_comment_len != comment_len) || (!mz_zip_string_equal(pComment, pFile_comment, file_comment_len, flags))) + continue; + } + if ((flags & MZ_ZIP_FLAG_IGNORE_PATH) && (filename_len)) + { + int ofs = filename_len - 1; + do + { + if ((pFilename[ofs] == '/') || (pFilename[ofs] == '\\') || (pFilename[ofs] == ':')) + break; + } while (--ofs >= 0); + ofs++; + pFilename += ofs; + filename_len -= ofs; + } + if ((filename_len == name_len) && (mz_zip_string_equal(pName, pFilename, filename_len, flags))) + { + if (pIndex) + *pIndex = file_index; + return MZ_TRUE; + } + } + + return mz_zip_set_error(pZip, MZ_ZIP_FILE_NOT_FOUND); + } + + static mz_bool mz_zip_reader_extract_to_mem_no_alloc1(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size, const mz_zip_archive_file_stat *st) + { + int status = TINFL_STATUS_DONE; + mz_uint64 needed_size, cur_file_ofs, comp_remaining, out_buf_ofs = 0, read_buf_size, read_buf_ofs = 0, read_buf_avail; + mz_zip_archive_file_stat file_stat; + void *pRead_buf; + mz_uint32 local_header_u32[(MZ_ZIP_LOCAL_DIR_HEADER_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pLocal_header = (mz_uint8 *)local_header_u32; + tinfl_decompressor inflator; + + if ((!pZip) || (!pZip->m_pState) || ((buf_size) && (!pBuf)) || ((user_read_buf_size) && (!pUser_read_buf)) || (!pZip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (st) + { + file_stat = *st; + } + else if (!mz_zip_reader_file_stat(pZip, file_index, &file_stat)) + return MZ_FALSE; + + /* A directory or zero length file */ + if ((file_stat.m_is_directory) || (!file_stat.m_comp_size)) + return MZ_TRUE; + + /* Encryption and patch files are not supported. */ + if (file_stat.m_bit_flag & (MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_IS_ENCRYPTED | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_USES_STRONG_ENCRYPTION | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_COMPRESSED_PATCH_FLAG)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_ENCRYPTION); + + /* This function only supports decompressing stored and deflate. */ + if ((!(flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) && (file_stat.m_method != 0) && (file_stat.m_method != MZ_DEFLATED)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_METHOD); + + /* Ensure supplied output buffer is large enough. */ + needed_size = (flags & MZ_ZIP_FLAG_COMPRESSED_DATA) ? file_stat.m_comp_size : file_stat.m_uncomp_size; + if (buf_size < needed_size) + return mz_zip_set_error(pZip, MZ_ZIP_BUF_TOO_SMALL); + + /* Read and parse the local directory entry. */ + cur_file_ofs = file_stat.m_local_header_ofs; + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pLocal_header, MZ_ZIP_LOCAL_DIR_HEADER_SIZE) != MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + + if (MZ_READ_LE32(pLocal_header) != MZ_ZIP_LOCAL_DIR_HEADER_SIG) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + cur_file_ofs += (mz_uint64)(MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_FILENAME_LEN_OFS) + MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_EXTRA_LEN_OFS); + if ((cur_file_ofs + file_stat.m_comp_size) > pZip->m_archive_size) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if ((flags & MZ_ZIP_FLAG_COMPRESSED_DATA) || (!file_stat.m_method)) + { + /* The file is stored or the caller has requested the compressed data. */ + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pBuf, (size_t)needed_size) != needed_size) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + if ((flags & MZ_ZIP_FLAG_COMPRESSED_DATA) == 0) + { + if (mz_crc32(MZ_CRC32_INIT, (const mz_uint8 *)pBuf, (size_t)file_stat.m_uncomp_size) != file_stat.m_crc32) + return mz_zip_set_error(pZip, MZ_ZIP_CRC_CHECK_FAILED); + } +#endif + + return MZ_TRUE; + } + + /* Decompress the file either directly from memory or from a file input buffer. */ + tinfl_init(&inflator); + + if (pZip->m_pState->m_pMem) + { + /* Read directly from the archive in memory. */ + pRead_buf = (mz_uint8 *)pZip->m_pState->m_pMem + cur_file_ofs; + read_buf_size = read_buf_avail = file_stat.m_comp_size; + comp_remaining = 0; + } + else if (pUser_read_buf) + { + /* Use a user provided read buffer. */ + if (!user_read_buf_size) + return MZ_FALSE; + pRead_buf = (mz_uint8 *)pUser_read_buf; + read_buf_size = user_read_buf_size; + read_buf_avail = 0; + comp_remaining = file_stat.m_comp_size; + } + else + { + /* Temporarily allocate a read buffer. */ + read_buf_size = MZ_MIN(file_stat.m_comp_size, (mz_uint64)MZ_ZIP_MAX_IO_BUF_SIZE); + if (((sizeof(size_t) == sizeof(mz_uint32))) && (read_buf_size > 0x7FFFFFFF)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if (NULL == (pRead_buf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, (size_t)read_buf_size))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + read_buf_avail = 0; + comp_remaining = file_stat.m_comp_size; + } + + do + { + /* The size_t cast here should be OK because we've verified that the output buffer is >= file_stat.m_uncomp_size above */ + size_t in_buf_size, out_buf_size = (size_t)(file_stat.m_uncomp_size - out_buf_ofs); + if ((!read_buf_avail) && (!pZip->m_pState->m_pMem)) + { + read_buf_avail = MZ_MIN(read_buf_size, comp_remaining); + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pRead_buf, (size_t)read_buf_avail) != read_buf_avail) + { + status = TINFL_STATUS_FAILED; + mz_zip_set_error(pZip, MZ_ZIP_DECOMPRESSION_FAILED); + break; + } + cur_file_ofs += read_buf_avail; + comp_remaining -= read_buf_avail; + read_buf_ofs = 0; + } + in_buf_size = (size_t)read_buf_avail; + status = tinfl_decompress(&inflator, (mz_uint8 *)pRead_buf + read_buf_ofs, &in_buf_size, (mz_uint8 *)pBuf, (mz_uint8 *)pBuf + out_buf_ofs, &out_buf_size, TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF | (comp_remaining ? TINFL_FLAG_HAS_MORE_INPUT : 0)); + read_buf_avail -= in_buf_size; + read_buf_ofs += in_buf_size; + out_buf_ofs += out_buf_size; + } while (status == TINFL_STATUS_NEEDS_MORE_INPUT); + + if (status == TINFL_STATUS_DONE) + { + /* Make sure the entire file was decompressed, and check its CRC. */ + if (out_buf_ofs != file_stat.m_uncomp_size) + { + mz_zip_set_error(pZip, MZ_ZIP_UNEXPECTED_DECOMPRESSED_SIZE); + status = TINFL_STATUS_FAILED; + } +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + else if (mz_crc32(MZ_CRC32_INIT, (const mz_uint8 *)pBuf, (size_t)file_stat.m_uncomp_size) != file_stat.m_crc32) + { + mz_zip_set_error(pZip, MZ_ZIP_CRC_CHECK_FAILED); + status = TINFL_STATUS_FAILED; + } +#endif + } + + if ((!pZip->m_pState->m_pMem) && (!pUser_read_buf)) + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + + return status == TINFL_STATUS_DONE; + } + + mz_bool mz_zip_reader_extract_to_mem_no_alloc(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size) + { + return mz_zip_reader_extract_to_mem_no_alloc1(pZip, file_index, pBuf, buf_size, flags, pUser_read_buf, user_read_buf_size, NULL); + } + + mz_bool mz_zip_reader_extract_file_to_mem_no_alloc(mz_zip_archive *pZip, const char *pFilename, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size) + { + mz_uint32 file_index; + if (!mz_zip_reader_locate_file_v2(pZip, pFilename, NULL, flags, &file_index)) + return MZ_FALSE; + return mz_zip_reader_extract_to_mem_no_alloc1(pZip, file_index, pBuf, buf_size, flags, pUser_read_buf, user_read_buf_size, NULL); + } + + mz_bool mz_zip_reader_extract_to_mem(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags) + { + return mz_zip_reader_extract_to_mem_no_alloc1(pZip, file_index, pBuf, buf_size, flags, NULL, 0, NULL); + } + + mz_bool mz_zip_reader_extract_file_to_mem(mz_zip_archive *pZip, const char *pFilename, void *pBuf, size_t buf_size, mz_uint flags) + { + return mz_zip_reader_extract_file_to_mem_no_alloc(pZip, pFilename, pBuf, buf_size, flags, NULL, 0); + } + + void *mz_zip_reader_extract_to_heap(mz_zip_archive *pZip, mz_uint file_index, size_t *pSize, mz_uint flags) + { + mz_zip_archive_file_stat file_stat; + mz_uint64 alloc_size; + void *pBuf; + + if (pSize) + *pSize = 0; + + if (!mz_zip_reader_file_stat(pZip, file_index, &file_stat)) + return NULL; + + alloc_size = (flags & MZ_ZIP_FLAG_COMPRESSED_DATA) ? file_stat.m_comp_size : file_stat.m_uncomp_size; + if (((sizeof(size_t) == sizeof(mz_uint32))) && (alloc_size > 0x7FFFFFFF)) + { + mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + return NULL; + } + + if (NULL == (pBuf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, (size_t)alloc_size))) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + return NULL; + } + + if (!mz_zip_reader_extract_to_mem_no_alloc1(pZip, file_index, pBuf, (size_t)alloc_size, flags, NULL, 0, &file_stat)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + return NULL; + } + + if (pSize) + *pSize = (size_t)alloc_size; + return pBuf; + } + + void *mz_zip_reader_extract_file_to_heap(mz_zip_archive *pZip, const char *pFilename, size_t *pSize, mz_uint flags) + { + mz_uint32 file_index; + if (!mz_zip_reader_locate_file_v2(pZip, pFilename, NULL, flags, &file_index)) + { + if (pSize) + *pSize = 0; + return MZ_FALSE; + } + return mz_zip_reader_extract_to_heap(pZip, file_index, pSize, flags); + } + + mz_bool mz_zip_reader_extract_to_callback(mz_zip_archive *pZip, mz_uint file_index, mz_file_write_func pCallback, void *pOpaque, mz_uint flags) + { + int status = TINFL_STATUS_DONE; +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + mz_uint file_crc32 = MZ_CRC32_INIT; +#endif + mz_uint64 read_buf_size, read_buf_ofs = 0, read_buf_avail, comp_remaining, out_buf_ofs = 0, cur_file_ofs; + mz_zip_archive_file_stat file_stat; + void *pRead_buf = NULL; + void *pWrite_buf = NULL; + mz_uint32 local_header_u32[(MZ_ZIP_LOCAL_DIR_HEADER_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pLocal_header = (mz_uint8 *)local_header_u32; + + if ((!pZip) || (!pZip->m_pState) || (!pCallback) || (!pZip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (!mz_zip_reader_file_stat(pZip, file_index, &file_stat)) + return MZ_FALSE; + + /* A directory or zero length file */ + if ((file_stat.m_is_directory) || (!file_stat.m_comp_size)) + return MZ_TRUE; + + /* Encryption and patch files are not supported. */ + if (file_stat.m_bit_flag & (MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_IS_ENCRYPTED | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_USES_STRONG_ENCRYPTION | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_COMPRESSED_PATCH_FLAG)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_ENCRYPTION); + + /* This function only supports decompressing stored and deflate. */ + if ((!(flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) && (file_stat.m_method != 0) && (file_stat.m_method != MZ_DEFLATED)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_METHOD); + + /* Read and do some minimal validation of the local directory entry (this doesn't crack the zip64 stuff, which we already have from the central dir) */ + cur_file_ofs = file_stat.m_local_header_ofs; + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pLocal_header, MZ_ZIP_LOCAL_DIR_HEADER_SIZE) != MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + + if (MZ_READ_LE32(pLocal_header) != MZ_ZIP_LOCAL_DIR_HEADER_SIG) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + cur_file_ofs += (mz_uint64)(MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_FILENAME_LEN_OFS) + MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_EXTRA_LEN_OFS); + if ((cur_file_ofs + file_stat.m_comp_size) > pZip->m_archive_size) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + /* Decompress the file either directly from memory or from a file input buffer. */ + if (pZip->m_pState->m_pMem) + { + pRead_buf = (mz_uint8 *)pZip->m_pState->m_pMem + cur_file_ofs; + read_buf_size = read_buf_avail = file_stat.m_comp_size; + comp_remaining = 0; + } + else + { + read_buf_size = MZ_MIN(file_stat.m_comp_size, (mz_uint64)MZ_ZIP_MAX_IO_BUF_SIZE); + if (NULL == (pRead_buf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, (size_t)read_buf_size))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + read_buf_avail = 0; + comp_remaining = file_stat.m_comp_size; + } + + if ((flags & MZ_ZIP_FLAG_COMPRESSED_DATA) || (!file_stat.m_method)) + { + /* The file is stored or the caller has requested the compressed data. */ + if (pZip->m_pState->m_pMem) + { + if (((sizeof(size_t) == sizeof(mz_uint32))) && (file_stat.m_comp_size > MZ_UINT32_MAX)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if (pCallback(pOpaque, out_buf_ofs, pRead_buf, (size_t)file_stat.m_comp_size) != file_stat.m_comp_size) + { + mz_zip_set_error(pZip, MZ_ZIP_WRITE_CALLBACK_FAILED); + status = TINFL_STATUS_FAILED; + } + else if (!(flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) + { +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + file_crc32 = (mz_uint32)mz_crc32(file_crc32, (const mz_uint8 *)pRead_buf, (size_t)file_stat.m_comp_size); +#endif + } + + cur_file_ofs += file_stat.m_comp_size; + out_buf_ofs += file_stat.m_comp_size; + comp_remaining = 0; + } + else + { + while (comp_remaining) + { + read_buf_avail = MZ_MIN(read_buf_size, comp_remaining); + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pRead_buf, (size_t)read_buf_avail) != read_buf_avail) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + status = TINFL_STATUS_FAILED; + break; + } + +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + if (!(flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) + { + file_crc32 = (mz_uint32)mz_crc32(file_crc32, (const mz_uint8 *)pRead_buf, (size_t)read_buf_avail); + } +#endif + + if (pCallback(pOpaque, out_buf_ofs, pRead_buf, (size_t)read_buf_avail) != read_buf_avail) + { + mz_zip_set_error(pZip, MZ_ZIP_WRITE_CALLBACK_FAILED); + status = TINFL_STATUS_FAILED; + break; + } + + cur_file_ofs += read_buf_avail; + out_buf_ofs += read_buf_avail; + comp_remaining -= read_buf_avail; + } + } + } + else + { + tinfl_decompressor inflator; + tinfl_init(&inflator); + + if (NULL == (pWrite_buf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, TINFL_LZ_DICT_SIZE))) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + status = TINFL_STATUS_FAILED; + } + else + { + do + { + mz_uint8 *pWrite_buf_cur = (mz_uint8 *)pWrite_buf + (out_buf_ofs & (TINFL_LZ_DICT_SIZE - 1)); + size_t in_buf_size, out_buf_size = TINFL_LZ_DICT_SIZE - (out_buf_ofs & (TINFL_LZ_DICT_SIZE - 1)); + if ((!read_buf_avail) && (!pZip->m_pState->m_pMem)) + { + read_buf_avail = MZ_MIN(read_buf_size, comp_remaining); + if (pZip->m_pRead(pZip->m_pIO_opaque, cur_file_ofs, pRead_buf, (size_t)read_buf_avail) != read_buf_avail) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + status = TINFL_STATUS_FAILED; + break; + } + cur_file_ofs += read_buf_avail; + comp_remaining -= read_buf_avail; + read_buf_ofs = 0; + } + + in_buf_size = (size_t)read_buf_avail; + status = tinfl_decompress(&inflator, (const mz_uint8 *)pRead_buf + read_buf_ofs, &in_buf_size, (mz_uint8 *)pWrite_buf, pWrite_buf_cur, &out_buf_size, comp_remaining ? TINFL_FLAG_HAS_MORE_INPUT : 0); + read_buf_avail -= in_buf_size; + read_buf_ofs += in_buf_size; + + if (out_buf_size) + { + if (pCallback(pOpaque, out_buf_ofs, pWrite_buf_cur, out_buf_size) != out_buf_size) + { + mz_zip_set_error(pZip, MZ_ZIP_WRITE_CALLBACK_FAILED); + status = TINFL_STATUS_FAILED; + break; + } + +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + file_crc32 = (mz_uint32)mz_crc32(file_crc32, pWrite_buf_cur, out_buf_size); +#endif + if ((out_buf_ofs += out_buf_size) > file_stat.m_uncomp_size) + { + mz_zip_set_error(pZip, MZ_ZIP_DECOMPRESSION_FAILED); + status = TINFL_STATUS_FAILED; + break; + } + } + } while ((status == TINFL_STATUS_NEEDS_MORE_INPUT) || (status == TINFL_STATUS_HAS_MORE_OUTPUT)); + } + } + + if ((status == TINFL_STATUS_DONE) && (!(flags & MZ_ZIP_FLAG_COMPRESSED_DATA))) + { + /* Make sure the entire file was decompressed, and check its CRC. */ + if (out_buf_ofs != file_stat.m_uncomp_size) + { + mz_zip_set_error(pZip, MZ_ZIP_UNEXPECTED_DECOMPRESSED_SIZE); + status = TINFL_STATUS_FAILED; + } +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + else if (file_crc32 != file_stat.m_crc32) + { + mz_zip_set_error(pZip, MZ_ZIP_DECOMPRESSION_FAILED); + status = TINFL_STATUS_FAILED; + } +#endif + } + + if (!pZip->m_pState->m_pMem) + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + + if (pWrite_buf) + pZip->m_pFree(pZip->m_pAlloc_opaque, pWrite_buf); + + return status == TINFL_STATUS_DONE; + } + + mz_bool mz_zip_reader_extract_file_to_callback(mz_zip_archive *pZip, const char *pFilename, mz_file_write_func pCallback, void *pOpaque, mz_uint flags) + { + mz_uint32 file_index; + if (!mz_zip_reader_locate_file_v2(pZip, pFilename, NULL, flags, &file_index)) + return MZ_FALSE; + + return mz_zip_reader_extract_to_callback(pZip, file_index, pCallback, pOpaque, flags); + } + + mz_zip_reader_extract_iter_state *mz_zip_reader_extract_iter_new(mz_zip_archive *pZip, mz_uint file_index, mz_uint flags) + { + mz_zip_reader_extract_iter_state *pState; + mz_uint32 local_header_u32[(MZ_ZIP_LOCAL_DIR_HEADER_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pLocal_header = (mz_uint8 *)local_header_u32; + + /* Argument sanity check */ + if ((!pZip) || (!pZip->m_pState)) + return NULL; + + /* Allocate an iterator status structure */ + pState = (mz_zip_reader_extract_iter_state *)pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, sizeof(mz_zip_reader_extract_iter_state)); + if (!pState) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + return NULL; + } + + /* Fetch file details */ + if (!mz_zip_reader_file_stat(pZip, file_index, &pState->file_stat)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + + /* Encryption and patch files are not supported. */ + if (pState->file_stat.m_bit_flag & (MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_IS_ENCRYPTED | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_USES_STRONG_ENCRYPTION | MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_COMPRESSED_PATCH_FLAG)) + { + mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_ENCRYPTION); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + + /* This function only supports decompressing stored and deflate. */ + if ((!(flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) && (pState->file_stat.m_method != 0) && (pState->file_stat.m_method != MZ_DEFLATED)) + { + mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_METHOD); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + + /* Init state - save args */ + pState->pZip = pZip; + pState->flags = flags; + + /* Init state - reset variables to defaults */ + pState->status = TINFL_STATUS_DONE; +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + pState->file_crc32 = MZ_CRC32_INIT; +#endif + pState->read_buf_ofs = 0; + pState->out_buf_ofs = 0; + pState->pRead_buf = NULL; + pState->pWrite_buf = NULL; + pState->out_blk_remain = 0; + + /* Read and parse the local directory entry. */ + pState->cur_file_ofs = pState->file_stat.m_local_header_ofs; + if (pZip->m_pRead(pZip->m_pIO_opaque, pState->cur_file_ofs, pLocal_header, MZ_ZIP_LOCAL_DIR_HEADER_SIZE) != MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + + if (MZ_READ_LE32(pLocal_header) != MZ_ZIP_LOCAL_DIR_HEADER_SIG) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + + pState->cur_file_ofs += (mz_uint64)(MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_FILENAME_LEN_OFS) + MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_EXTRA_LEN_OFS); + if ((pState->cur_file_ofs + pState->file_stat.m_comp_size) > pZip->m_archive_size) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + + /* Decompress the file either directly from memory or from a file input buffer. */ + if (pZip->m_pState->m_pMem) + { + pState->pRead_buf = (mz_uint8 *)pZip->m_pState->m_pMem + pState->cur_file_ofs; + pState->read_buf_size = pState->read_buf_avail = pState->file_stat.m_comp_size; + pState->comp_remaining = pState->file_stat.m_comp_size; + } + else + { + if (!((flags & MZ_ZIP_FLAG_COMPRESSED_DATA) || (!pState->file_stat.m_method))) + { + /* Decompression required, therefore intermediate read buffer required */ + pState->read_buf_size = MZ_MIN(pState->file_stat.m_comp_size, (mz_uint64)MZ_ZIP_MAX_IO_BUF_SIZE); + if (NULL == (pState->pRead_buf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, (size_t)pState->read_buf_size))) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + } + else + { + /* Decompression not required - we will be reading directly into user buffer, no temp buf required */ + pState->read_buf_size = 0; + } + pState->read_buf_avail = 0; + pState->comp_remaining = pState->file_stat.m_comp_size; + } + + if (!((flags & MZ_ZIP_FLAG_COMPRESSED_DATA) || (!pState->file_stat.m_method))) + { + /* Decompression required, init decompressor */ + tinfl_init(&pState->inflator); + + /* Allocate write buffer */ + if (NULL == (pState->pWrite_buf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, TINFL_LZ_DICT_SIZE))) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + if (pState->pRead_buf) + pZip->m_pFree(pZip->m_pAlloc_opaque, pState->pRead_buf); + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + return NULL; + } + } + + return pState; + } + + mz_zip_reader_extract_iter_state *mz_zip_reader_extract_file_iter_new(mz_zip_archive *pZip, const char *pFilename, mz_uint flags) + { + mz_uint32 file_index; + + /* Locate file index by name */ + if (!mz_zip_reader_locate_file_v2(pZip, pFilename, NULL, flags, &file_index)) + return NULL; + + /* Construct iterator */ + return mz_zip_reader_extract_iter_new(pZip, file_index, flags); + } + + size_t mz_zip_reader_extract_iter_read(mz_zip_reader_extract_iter_state *pState, void *pvBuf, size_t buf_size) + { + size_t copied_to_caller = 0; + + /* Argument sanity check */ + if ((!pState) || (!pState->pZip) || (!pState->pZip->m_pState) || (!pvBuf)) + return 0; + + if ((pState->flags & MZ_ZIP_FLAG_COMPRESSED_DATA) || (!pState->file_stat.m_method)) + { + /* The file is stored or the caller has requested the compressed data, calc amount to return. */ + copied_to_caller = (size_t)MZ_MIN(buf_size, pState->comp_remaining); + + /* Zip is in memory....or requires reading from a file? */ + if (pState->pZip->m_pState->m_pMem) + { + /* Copy data to caller's buffer */ + memcpy(pvBuf, pState->pRead_buf, copied_to_caller); + pState->pRead_buf = ((mz_uint8 *)pState->pRead_buf) + copied_to_caller; + } + else + { + /* Read directly into caller's buffer */ + if (pState->pZip->m_pRead(pState->pZip->m_pIO_opaque, pState->cur_file_ofs, pvBuf, copied_to_caller) != copied_to_caller) + { + /* Failed to read all that was asked for, flag failure and alert user */ + mz_zip_set_error(pState->pZip, MZ_ZIP_FILE_READ_FAILED); + pState->status = TINFL_STATUS_FAILED; + copied_to_caller = 0; + } + } + +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + /* Compute CRC if not returning compressed data only */ + if (!(pState->flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) + pState->file_crc32 = (mz_uint32)mz_crc32(pState->file_crc32, (const mz_uint8 *)pvBuf, copied_to_caller); +#endif + + /* Advance offsets, dec counters */ + pState->cur_file_ofs += copied_to_caller; + pState->out_buf_ofs += copied_to_caller; + pState->comp_remaining -= copied_to_caller; + } + else + { + do + { + /* Calc ptr to write buffer - given current output pos and block size */ + mz_uint8 *pWrite_buf_cur = (mz_uint8 *)pState->pWrite_buf + (pState->out_buf_ofs & (TINFL_LZ_DICT_SIZE - 1)); + + /* Calc max output size - given current output pos and block size */ + size_t in_buf_size, out_buf_size = TINFL_LZ_DICT_SIZE - (pState->out_buf_ofs & (TINFL_LZ_DICT_SIZE - 1)); + + if (!pState->out_blk_remain) + { + /* Read more data from file if none available (and reading from file) */ + if ((!pState->read_buf_avail) && (!pState->pZip->m_pState->m_pMem)) + { + /* Calc read size */ + pState->read_buf_avail = MZ_MIN(pState->read_buf_size, pState->comp_remaining); + if (pState->pZip->m_pRead(pState->pZip->m_pIO_opaque, pState->cur_file_ofs, pState->pRead_buf, (size_t)pState->read_buf_avail) != pState->read_buf_avail) + { + mz_zip_set_error(pState->pZip, MZ_ZIP_FILE_READ_FAILED); + pState->status = TINFL_STATUS_FAILED; + break; + } + + /* Advance offsets, dec counters */ + pState->cur_file_ofs += pState->read_buf_avail; + pState->comp_remaining -= pState->read_buf_avail; + pState->read_buf_ofs = 0; + } + + /* Perform decompression */ + in_buf_size = (size_t)pState->read_buf_avail; + pState->status = tinfl_decompress(&pState->inflator, (const mz_uint8 *)pState->pRead_buf + pState->read_buf_ofs, &in_buf_size, (mz_uint8 *)pState->pWrite_buf, pWrite_buf_cur, &out_buf_size, pState->comp_remaining ? TINFL_FLAG_HAS_MORE_INPUT : 0); + pState->read_buf_avail -= in_buf_size; + pState->read_buf_ofs += in_buf_size; + + /* Update current output block size remaining */ + pState->out_blk_remain = out_buf_size; + } + + if (pState->out_blk_remain) + { + /* Calc amount to return. */ + size_t to_copy = MZ_MIN((buf_size - copied_to_caller), pState->out_blk_remain); + + /* Copy data to caller's buffer */ + memcpy((mz_uint8 *)pvBuf + copied_to_caller, pWrite_buf_cur, to_copy); + +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + /* Perform CRC */ + pState->file_crc32 = (mz_uint32)mz_crc32(pState->file_crc32, pWrite_buf_cur, to_copy); +#endif + + /* Decrement data consumed from block */ + pState->out_blk_remain -= to_copy; + + /* Inc output offset, while performing sanity check */ + if ((pState->out_buf_ofs += to_copy) > pState->file_stat.m_uncomp_size) + { + mz_zip_set_error(pState->pZip, MZ_ZIP_DECOMPRESSION_FAILED); + pState->status = TINFL_STATUS_FAILED; + break; + } + + /* Increment counter of data copied to caller */ + copied_to_caller += to_copy; + } + } while ((copied_to_caller < buf_size) && ((pState->status == TINFL_STATUS_NEEDS_MORE_INPUT) || (pState->status == TINFL_STATUS_HAS_MORE_OUTPUT))); + } + + /* Return how many bytes were copied into user buffer */ + return copied_to_caller; + } + + mz_bool mz_zip_reader_extract_iter_free(mz_zip_reader_extract_iter_state *pState) + { + int status; + + /* Argument sanity check */ + if ((!pState) || (!pState->pZip) || (!pState->pZip->m_pState)) + return MZ_FALSE; + + /* Was decompression completed and requested? */ + if ((pState->status == TINFL_STATUS_DONE) && (!(pState->flags & MZ_ZIP_FLAG_COMPRESSED_DATA))) + { + /* Make sure the entire file was decompressed, and check its CRC. */ + if (pState->out_buf_ofs != pState->file_stat.m_uncomp_size) + { + mz_zip_set_error(pState->pZip, MZ_ZIP_UNEXPECTED_DECOMPRESSED_SIZE); + pState->status = TINFL_STATUS_FAILED; + } +#ifndef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + else if (pState->file_crc32 != pState->file_stat.m_crc32) + { + mz_zip_set_error(pState->pZip, MZ_ZIP_DECOMPRESSION_FAILED); + pState->status = TINFL_STATUS_FAILED; + } +#endif + } + + /* Free buffers */ + if (!pState->pZip->m_pState->m_pMem) + pState->pZip->m_pFree(pState->pZip->m_pAlloc_opaque, pState->pRead_buf); + if (pState->pWrite_buf) + pState->pZip->m_pFree(pState->pZip->m_pAlloc_opaque, pState->pWrite_buf); + + /* Save status */ + status = pState->status; + + /* Free context */ + pState->pZip->m_pFree(pState->pZip->m_pAlloc_opaque, pState); + + return status == TINFL_STATUS_DONE; + } + +#ifndef MINIZ_NO_STDIO + static size_t mz_zip_file_write_callback(void *pOpaque, mz_uint64 ofs, const void *pBuf, size_t n) + { + (void)ofs; + + return MZ_FWRITE(pBuf, 1, n, (MZ_FILE *)pOpaque); + } + + mz_bool mz_zip_reader_extract_to_file(mz_zip_archive *pZip, mz_uint file_index, const char *pDst_filename, mz_uint flags) + { + mz_bool status; + mz_zip_archive_file_stat file_stat; + MZ_FILE *pFile; + + if (!mz_zip_reader_file_stat(pZip, file_index, &file_stat)) + return MZ_FALSE; + + if ((file_stat.m_is_directory) || (!file_stat.m_is_supported)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_FEATURE); + + pFile = MZ_FOPEN(pDst_filename, "wb"); + if (!pFile) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_OPEN_FAILED); + + status = mz_zip_reader_extract_to_callback(pZip, file_index, mz_zip_file_write_callback, pFile, flags); + + if (MZ_FCLOSE(pFile) == EOF) + { + if (status) + mz_zip_set_error(pZip, MZ_ZIP_FILE_CLOSE_FAILED); + + status = MZ_FALSE; + } + +#if !defined(MINIZ_NO_TIME) && !defined(MINIZ_NO_STDIO) + if (status) + mz_zip_set_file_times(pDst_filename, file_stat.m_time, file_stat.m_time); +#endif + + return status; + } + + mz_bool mz_zip_reader_extract_file_to_file(mz_zip_archive *pZip, const char *pArchive_filename, const char *pDst_filename, mz_uint flags) + { + mz_uint32 file_index; + if (!mz_zip_reader_locate_file_v2(pZip, pArchive_filename, NULL, flags, &file_index)) + return MZ_FALSE; + + return mz_zip_reader_extract_to_file(pZip, file_index, pDst_filename, flags); + } + + mz_bool mz_zip_reader_extract_to_cfile(mz_zip_archive *pZip, mz_uint file_index, MZ_FILE *pFile, mz_uint flags) + { + mz_zip_archive_file_stat file_stat; + + if (!mz_zip_reader_file_stat(pZip, file_index, &file_stat)) + return MZ_FALSE; + + if ((file_stat.m_is_directory) || (!file_stat.m_is_supported)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_FEATURE); + + return mz_zip_reader_extract_to_callback(pZip, file_index, mz_zip_file_write_callback, pFile, flags); + } + + mz_bool mz_zip_reader_extract_file_to_cfile(mz_zip_archive *pZip, const char *pArchive_filename, MZ_FILE *pFile, mz_uint flags) + { + mz_uint32 file_index; + if (!mz_zip_reader_locate_file_v2(pZip, pArchive_filename, NULL, flags, &file_index)) + return MZ_FALSE; + + return mz_zip_reader_extract_to_cfile(pZip, file_index, pFile, flags); + } +#endif /* #ifndef MINIZ_NO_STDIO */ + + static size_t mz_zip_compute_crc32_callback(void *pOpaque, mz_uint64 file_ofs, const void *pBuf, size_t n) + { + mz_uint32 *p = (mz_uint32 *)pOpaque; + (void)file_ofs; + *p = (mz_uint32)mz_crc32(*p, (const mz_uint8 *)pBuf, n); + return n; + } + + mz_bool mz_zip_validate_file(mz_zip_archive *pZip, mz_uint file_index, mz_uint flags) + { + mz_zip_archive_file_stat file_stat; + mz_zip_internal_state *pState; + const mz_uint8 *pCentral_dir_header; + mz_bool found_zip64_ext_data_in_cdir = MZ_FALSE; + mz_bool found_zip64_ext_data_in_ldir = MZ_FALSE; + mz_uint32 local_header_u32[(MZ_ZIP_LOCAL_DIR_HEADER_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pLocal_header = (mz_uint8 *)local_header_u32; + mz_uint64 local_header_ofs = 0; + mz_uint32 local_header_filename_len, local_header_extra_len, local_header_crc32; + mz_uint64 local_header_comp_size, local_header_uncomp_size; + mz_uint32 uncomp_crc32 = MZ_CRC32_INIT; + mz_bool has_data_descriptor; + mz_uint32 local_header_bit_flags; + + mz_zip_array file_data_array; + mz_zip_array_init(&file_data_array, 1); + + if ((!pZip) || (!pZip->m_pState) || (!pZip->m_pAlloc) || (!pZip->m_pFree) || (!pZip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (file_index > pZip->m_total_files) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState = pZip->m_pState; + + pCentral_dir_header = mz_zip_get_cdh(pZip, file_index); + + if (!mz_zip_file_stat_internal(pZip, file_index, pCentral_dir_header, &file_stat, &found_zip64_ext_data_in_cdir)) + return MZ_FALSE; + + /* A directory or zero length file */ + if ((file_stat.m_is_directory) || (!file_stat.m_uncomp_size)) + return MZ_TRUE; + + /* Encryption and patch files are not supported. */ + if (file_stat.m_is_encrypted) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_ENCRYPTION); + + /* This function only supports stored and deflate. */ + if ((file_stat.m_method != 0) && (file_stat.m_method != MZ_DEFLATED)) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_METHOD); + + if (!file_stat.m_is_supported) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_FEATURE); + + /* Read and parse the local directory entry. */ + local_header_ofs = file_stat.m_local_header_ofs; + if (pZip->m_pRead(pZip->m_pIO_opaque, local_header_ofs, pLocal_header, MZ_ZIP_LOCAL_DIR_HEADER_SIZE) != MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + + if (MZ_READ_LE32(pLocal_header) != MZ_ZIP_LOCAL_DIR_HEADER_SIG) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + local_header_filename_len = MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_FILENAME_LEN_OFS); + local_header_extra_len = MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_EXTRA_LEN_OFS); + local_header_comp_size = MZ_READ_LE32(pLocal_header + MZ_ZIP_LDH_COMPRESSED_SIZE_OFS); + local_header_uncomp_size = MZ_READ_LE32(pLocal_header + MZ_ZIP_LDH_DECOMPRESSED_SIZE_OFS); + local_header_crc32 = MZ_READ_LE32(pLocal_header + MZ_ZIP_LDH_CRC32_OFS); + local_header_bit_flags = MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_BIT_FLAG_OFS); + has_data_descriptor = (local_header_bit_flags & 8) != 0; + + if (local_header_filename_len != strlen(file_stat.m_filename)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if ((local_header_ofs + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + local_header_filename_len + local_header_extra_len + file_stat.m_comp_size) > pZip->m_archive_size) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if (!mz_zip_array_resize(pZip, &file_data_array, MZ_MAX(local_header_filename_len, local_header_extra_len), MZ_FALSE)) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + goto handle_failure; + } + + if (local_header_filename_len) + { + if (pZip->m_pRead(pZip->m_pIO_opaque, local_header_ofs + MZ_ZIP_LOCAL_DIR_HEADER_SIZE, file_data_array.m_p, local_header_filename_len) != local_header_filename_len) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + goto handle_failure; + } + + /* I've seen 1 archive that had the same pathname, but used backslashes in the local dir and forward slashes in the central dir. Do we care about this? For now, this case will fail validation. */ + if (memcmp(file_stat.m_filename, file_data_array.m_p, local_header_filename_len) != 0) + { + mz_zip_set_error(pZip, MZ_ZIP_VALIDATION_FAILED); + goto handle_failure; + } + } + + if ((local_header_extra_len) && ((local_header_comp_size == MZ_UINT32_MAX) || (local_header_uncomp_size == MZ_UINT32_MAX))) + { + mz_uint32 extra_size_remaining = local_header_extra_len; + const mz_uint8 *pExtra_data = (const mz_uint8 *)file_data_array.m_p; + + if (pZip->m_pRead(pZip->m_pIO_opaque, local_header_ofs + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + local_header_filename_len, file_data_array.m_p, local_header_extra_len) != local_header_extra_len) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + goto handle_failure; + } + + do + { + mz_uint32 field_id, field_data_size, field_total_size; + + if (extra_size_remaining < (sizeof(mz_uint16) * 2)) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + goto handle_failure; + } + + field_id = MZ_READ_LE16(pExtra_data); + field_data_size = MZ_READ_LE16(pExtra_data + sizeof(mz_uint16)); + field_total_size = field_data_size + sizeof(mz_uint16) * 2; + + if (field_total_size > extra_size_remaining) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + goto handle_failure; + } + + if (field_id == MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID) + { + const mz_uint8 *pSrc_field_data = pExtra_data + sizeof(mz_uint32); + + if (field_data_size < sizeof(mz_uint64) * 2) + { + mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + goto handle_failure; + } + + local_header_uncomp_size = MZ_READ_LE64(pSrc_field_data); + local_header_comp_size = MZ_READ_LE64(pSrc_field_data + sizeof(mz_uint64)); + + found_zip64_ext_data_in_ldir = MZ_TRUE; + break; + } + + pExtra_data += field_total_size; + extra_size_remaining -= field_total_size; + } while (extra_size_remaining); + } + + /* TODO: parse local header extra data when local_header_comp_size is 0xFFFFFFFF! (big_descriptor.zip) */ + /* I've seen zips in the wild with the data descriptor bit set, but proper local header values and bogus data descriptors */ + if ((has_data_descriptor) && (!local_header_comp_size) && (!local_header_crc32)) + { + mz_uint8 descriptor_buf[32]; + mz_bool has_id; + const mz_uint8 *pSrc; + mz_uint32 file_crc32; + mz_uint64 comp_size = 0, uncomp_size = 0; + + mz_uint32 num_descriptor_uint32s = ((pState->m_zip64) || (found_zip64_ext_data_in_ldir)) ? 6 : 4; + + if (pZip->m_pRead(pZip->m_pIO_opaque, local_header_ofs + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + local_header_filename_len + local_header_extra_len + file_stat.m_comp_size, descriptor_buf, sizeof(mz_uint32) * num_descriptor_uint32s) != (sizeof(mz_uint32) * num_descriptor_uint32s)) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + goto handle_failure; + } + + has_id = (MZ_READ_LE32(descriptor_buf) == MZ_ZIP_DATA_DESCRIPTOR_ID); + pSrc = has_id ? (descriptor_buf + sizeof(mz_uint32)) : descriptor_buf; + + file_crc32 = MZ_READ_LE32(pSrc); + + if ((pState->m_zip64) || (found_zip64_ext_data_in_ldir)) + { + comp_size = MZ_READ_LE64(pSrc + sizeof(mz_uint32)); + uncomp_size = MZ_READ_LE64(pSrc + sizeof(mz_uint32) + sizeof(mz_uint64)); + } + else + { + comp_size = MZ_READ_LE32(pSrc + sizeof(mz_uint32)); + uncomp_size = MZ_READ_LE32(pSrc + sizeof(mz_uint32) + sizeof(mz_uint32)); + } + + if ((file_crc32 != file_stat.m_crc32) || (comp_size != file_stat.m_comp_size) || (uncomp_size != file_stat.m_uncomp_size)) + { + mz_zip_set_error(pZip, MZ_ZIP_VALIDATION_FAILED); + goto handle_failure; + } + } + else + { + if ((local_header_crc32 != file_stat.m_crc32) || (local_header_comp_size != file_stat.m_comp_size) || (local_header_uncomp_size != file_stat.m_uncomp_size)) + { + mz_zip_set_error(pZip, MZ_ZIP_VALIDATION_FAILED); + goto handle_failure; + } + } + + mz_zip_array_clear(pZip, &file_data_array); + + if ((flags & MZ_ZIP_FLAG_VALIDATE_HEADERS_ONLY) == 0) + { + if (!mz_zip_reader_extract_to_callback(pZip, file_index, mz_zip_compute_crc32_callback, &uncomp_crc32, 0)) + return MZ_FALSE; + + /* 1 more check to be sure, although the extract checks too. */ + if (uncomp_crc32 != file_stat.m_crc32) + { + mz_zip_set_error(pZip, MZ_ZIP_VALIDATION_FAILED); + return MZ_FALSE; + } + } + + return MZ_TRUE; + + handle_failure: + mz_zip_array_clear(pZip, &file_data_array); + return MZ_FALSE; + } + + mz_bool mz_zip_validate_archive(mz_zip_archive *pZip, mz_uint flags) + { + mz_zip_internal_state *pState; + mz_uint32 i; + + if ((!pZip) || (!pZip->m_pState) || (!pZip->m_pAlloc) || (!pZip->m_pFree) || (!pZip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState = pZip->m_pState; + + /* Basic sanity checks */ + if (!pState->m_zip64) + { + if (pZip->m_total_files > MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + + if (pZip->m_archive_size > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + } + else + { + if (pState->m_central_dir.m_size >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + } + + for (i = 0; i < pZip->m_total_files; i++) + { + if (MZ_ZIP_FLAG_VALIDATE_LOCATE_FILE_FLAG & flags) + { + mz_uint32 found_index; + mz_zip_archive_file_stat stat; + + if (!mz_zip_reader_file_stat(pZip, i, &stat)) + return MZ_FALSE; + + if (!mz_zip_reader_locate_file_v2(pZip, stat.m_filename, NULL, 0, &found_index)) + return MZ_FALSE; + + /* This check can fail if there are duplicate filenames in the archive (which we don't check for when writing - that's up to the user) */ + if (found_index != i) + return mz_zip_set_error(pZip, MZ_ZIP_VALIDATION_FAILED); + } + + if (!mz_zip_validate_file(pZip, i, flags)) + return MZ_FALSE; + } + + return MZ_TRUE; + } + + mz_bool mz_zip_validate_mem_archive(const void *pMem, size_t size, mz_uint flags, mz_zip_error *pErr) + { + mz_bool success = MZ_TRUE; + mz_zip_archive zip; + mz_zip_error actual_err = MZ_ZIP_NO_ERROR; + + if ((!pMem) || (!size)) + { + if (pErr) + *pErr = MZ_ZIP_INVALID_PARAMETER; + return MZ_FALSE; + } + + mz_zip_zero_struct(&zip); + + if (!mz_zip_reader_init_mem(&zip, pMem, size, flags)) + { + if (pErr) + *pErr = zip.m_last_error; + return MZ_FALSE; + } + + if (!mz_zip_validate_archive(&zip, flags)) + { + actual_err = zip.m_last_error; + success = MZ_FALSE; + } + + if (!mz_zip_reader_end_internal(&zip, success)) + { + if (!actual_err) + actual_err = zip.m_last_error; + success = MZ_FALSE; + } + + if (pErr) + *pErr = actual_err; + + return success; + } + +#ifndef MINIZ_NO_STDIO + mz_bool mz_zip_validate_file_archive(const char *pFilename, mz_uint flags, mz_zip_error *pErr) + { + mz_bool success = MZ_TRUE; + mz_zip_archive zip; + mz_zip_error actual_err = MZ_ZIP_NO_ERROR; + + if (!pFilename) + { + if (pErr) + *pErr = MZ_ZIP_INVALID_PARAMETER; + return MZ_FALSE; + } + + mz_zip_zero_struct(&zip); + + if (!mz_zip_reader_init_file_v2(&zip, pFilename, flags, 0, 0)) + { + if (pErr) + *pErr = zip.m_last_error; + return MZ_FALSE; + } + + if (!mz_zip_validate_archive(&zip, flags)) + { + actual_err = zip.m_last_error; + success = MZ_FALSE; + } + + if (!mz_zip_reader_end_internal(&zip, success)) + { + if (!actual_err) + actual_err = zip.m_last_error; + success = MZ_FALSE; + } + + if (pErr) + *pErr = actual_err; + + return success; + } +#endif /* #ifndef MINIZ_NO_STDIO */ + + /* ------------------- .ZIP archive writing */ + +#ifndef MINIZ_NO_ARCHIVE_WRITING_APIS + + static MZ_FORCEINLINE void mz_write_le16(mz_uint8 *p, mz_uint16 v) + { + p[0] = (mz_uint8)v; + p[1] = (mz_uint8)(v >> 8); + } + static MZ_FORCEINLINE void mz_write_le32(mz_uint8 *p, mz_uint32 v) + { + p[0] = (mz_uint8)v; + p[1] = (mz_uint8)(v >> 8); + p[2] = (mz_uint8)(v >> 16); + p[3] = (mz_uint8)(v >> 24); + } + static MZ_FORCEINLINE void mz_write_le64(mz_uint8 *p, mz_uint64 v) + { + mz_write_le32(p, (mz_uint32)v); + mz_write_le32(p + sizeof(mz_uint32), (mz_uint32)(v >> 32)); + } + +#define MZ_WRITE_LE16(p, v) mz_write_le16((mz_uint8 *)(p), (mz_uint16)(v)) +#define MZ_WRITE_LE32(p, v) mz_write_le32((mz_uint8 *)(p), (mz_uint32)(v)) +#define MZ_WRITE_LE64(p, v) mz_write_le64((mz_uint8 *)(p), (mz_uint64)(v)) + + static size_t mz_zip_heap_write_func(void *pOpaque, mz_uint64 file_ofs, const void *pBuf, size_t n) + { + mz_zip_archive *pZip = (mz_zip_archive *)pOpaque; + mz_zip_internal_state *pState = pZip->m_pState; + mz_uint64 new_size = MZ_MAX(file_ofs + n, pState->m_mem_size); + + if (!n) + return 0; + + /* An allocation this big is likely to just fail on 32-bit systems, so don't even go there. */ + if ((sizeof(size_t) == sizeof(mz_uint32)) && (new_size > 0x7FFFFFFF)) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_TOO_LARGE); + return 0; + } + + if (new_size > pState->m_mem_capacity) + { + void *pNew_block; + size_t new_capacity = MZ_MAX(64, pState->m_mem_capacity); + + while (new_capacity < new_size) + new_capacity *= 2; + + if (NULL == (pNew_block = pZip->m_pRealloc(pZip->m_pAlloc_opaque, pState->m_pMem, 1, new_capacity))) + { + mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + return 0; + } + + pState->m_pMem = pNew_block; + pState->m_mem_capacity = new_capacity; + } + memcpy((mz_uint8 *)pState->m_pMem + file_ofs, pBuf, n); + pState->m_mem_size = (size_t)new_size; + return n; + } + + static mz_bool mz_zip_writer_end_internal(mz_zip_archive *pZip, mz_bool set_last_error) + { + mz_zip_internal_state *pState; + mz_bool status = MZ_TRUE; + + if ((!pZip) || (!pZip->m_pState) || (!pZip->m_pAlloc) || (!pZip->m_pFree) || ((pZip->m_zip_mode != MZ_ZIP_MODE_WRITING) && (pZip->m_zip_mode != MZ_ZIP_MODE_WRITING_HAS_BEEN_FINALIZED))) + { + if (set_last_error) + mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + return MZ_FALSE; + } + + pState = pZip->m_pState; + pZip->m_pState = NULL; + mz_zip_array_clear(pZip, &pState->m_central_dir); + mz_zip_array_clear(pZip, &pState->m_central_dir_offsets); + mz_zip_array_clear(pZip, &pState->m_sorted_central_dir_offsets); + +#ifndef MINIZ_NO_STDIO + if (pState->m_pFile) + { + if (pZip->m_zip_type == MZ_ZIP_TYPE_FILE) + { + if (MZ_FCLOSE(pState->m_pFile) == EOF) + { + if (set_last_error) + mz_zip_set_error(pZip, MZ_ZIP_FILE_CLOSE_FAILED); + status = MZ_FALSE; + } + } + + pState->m_pFile = NULL; + } +#endif /* #ifndef MINIZ_NO_STDIO */ + + if ((pZip->m_pWrite == mz_zip_heap_write_func) && (pState->m_pMem)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pState->m_pMem); + pState->m_pMem = NULL; + } + + pZip->m_pFree(pZip->m_pAlloc_opaque, pState); + pZip->m_zip_mode = MZ_ZIP_MODE_INVALID; + return status; + } + + mz_bool mz_zip_writer_init_v2(mz_zip_archive *pZip, mz_uint64 existing_size, mz_uint flags) + { + mz_bool zip64 = (flags & MZ_ZIP_FLAG_WRITE_ZIP64) != 0; + + if ((!pZip) || (pZip->m_pState) || (!pZip->m_pWrite) || (pZip->m_zip_mode != MZ_ZIP_MODE_INVALID)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (flags & MZ_ZIP_FLAG_WRITE_ALLOW_READING) + { + if (!pZip->m_pRead) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + } + + if (pZip->m_file_offset_alignment) + { + /* Ensure user specified file offset alignment is a power of 2. */ + if (pZip->m_file_offset_alignment & (pZip->m_file_offset_alignment - 1)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + } + + if (!pZip->m_pAlloc) + pZip->m_pAlloc = miniz_def_alloc_func; + if (!pZip->m_pFree) + pZip->m_pFree = miniz_def_free_func; + if (!pZip->m_pRealloc) + pZip->m_pRealloc = miniz_def_realloc_func; + + pZip->m_archive_size = existing_size; + pZip->m_central_directory_file_ofs = 0; + pZip->m_total_files = 0; + + if (NULL == (pZip->m_pState = (mz_zip_internal_state *)pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, sizeof(mz_zip_internal_state)))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + memset(pZip->m_pState, 0, sizeof(mz_zip_internal_state)); + + MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(&pZip->m_pState->m_central_dir, sizeof(mz_uint8)); + MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(&pZip->m_pState->m_central_dir_offsets, sizeof(mz_uint32)); + MZ_ZIP_ARRAY_SET_ELEMENT_SIZE(&pZip->m_pState->m_sorted_central_dir_offsets, sizeof(mz_uint32)); + + pZip->m_pState->m_zip64 = zip64; + pZip->m_pState->m_zip64_has_extended_info_fields = zip64; + + pZip->m_zip_type = MZ_ZIP_TYPE_USER; + pZip->m_zip_mode = MZ_ZIP_MODE_WRITING; + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_init(mz_zip_archive *pZip, mz_uint64 existing_size) + { + return mz_zip_writer_init_v2(pZip, existing_size, 0); + } + + mz_bool mz_zip_writer_init_heap_v2(mz_zip_archive *pZip, size_t size_to_reserve_at_beginning, size_t initial_allocation_size, mz_uint flags) + { + pZip->m_pWrite = mz_zip_heap_write_func; + pZip->m_pNeeds_keepalive = NULL; + + if (flags & MZ_ZIP_FLAG_WRITE_ALLOW_READING) + pZip->m_pRead = mz_zip_mem_read_func; + + pZip->m_pIO_opaque = pZip; + + if (!mz_zip_writer_init_v2(pZip, size_to_reserve_at_beginning, flags)) + return MZ_FALSE; + + pZip->m_zip_type = MZ_ZIP_TYPE_HEAP; + + if (0 != (initial_allocation_size = MZ_MAX(initial_allocation_size, size_to_reserve_at_beginning))) + { + if (NULL == (pZip->m_pState->m_pMem = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, initial_allocation_size))) + { + mz_zip_writer_end_internal(pZip, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + pZip->m_pState->m_mem_capacity = initial_allocation_size; + } + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_init_heap(mz_zip_archive *pZip, size_t size_to_reserve_at_beginning, size_t initial_allocation_size) + { + return mz_zip_writer_init_heap_v2(pZip, size_to_reserve_at_beginning, initial_allocation_size, 0); + } + +#ifndef MINIZ_NO_STDIO + static size_t mz_zip_file_write_func(void *pOpaque, mz_uint64 file_ofs, const void *pBuf, size_t n) + { + mz_zip_archive *pZip = (mz_zip_archive *)pOpaque; + mz_int64 cur_ofs = MZ_FTELL64(pZip->m_pState->m_pFile); + + file_ofs += pZip->m_pState->m_file_archive_start_ofs; + + if (((mz_int64)file_ofs < 0) || (((cur_ofs != (mz_int64)file_ofs)) && (MZ_FSEEK64(pZip->m_pState->m_pFile, (mz_int64)file_ofs, SEEK_SET)))) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_SEEK_FAILED); + return 0; + } + + return MZ_FWRITE(pBuf, 1, n, pZip->m_pState->m_pFile); + } + + mz_bool mz_zip_writer_init_file(mz_zip_archive *pZip, const char *pFilename, mz_uint64 size_to_reserve_at_beginning) + { + return mz_zip_writer_init_file_v2(pZip, pFilename, size_to_reserve_at_beginning, 0); + } + + mz_bool mz_zip_writer_init_file_v2(mz_zip_archive *pZip, const char *pFilename, mz_uint64 size_to_reserve_at_beginning, mz_uint flags) + { + MZ_FILE *pFile; + + pZip->m_pWrite = mz_zip_file_write_func; + pZip->m_pNeeds_keepalive = NULL; + + if (flags & MZ_ZIP_FLAG_WRITE_ALLOW_READING) + pZip->m_pRead = mz_zip_file_read_func; + + pZip->m_pIO_opaque = pZip; + + if (!mz_zip_writer_init_v2(pZip, size_to_reserve_at_beginning, flags)) + return MZ_FALSE; + + if (NULL == (pFile = MZ_FOPEN(pFilename, (flags & MZ_ZIP_FLAG_WRITE_ALLOW_READING) ? "w+b" : "wb"))) + { + mz_zip_writer_end(pZip); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_OPEN_FAILED); + } + + pZip->m_pState->m_pFile = pFile; + pZip->m_zip_type = MZ_ZIP_TYPE_FILE; + + if (size_to_reserve_at_beginning) + { + mz_uint64 cur_ofs = 0; + char buf[4096]; + + MZ_CLEAR_ARR(buf); + + do + { + size_t n = (size_t)MZ_MIN(sizeof(buf), size_to_reserve_at_beginning); + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_ofs, buf, n) != n) + { + mz_zip_writer_end(pZip); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + cur_ofs += n; + size_to_reserve_at_beginning -= n; + } while (size_to_reserve_at_beginning); + } + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_init_cfile(mz_zip_archive *pZip, MZ_FILE *pFile, mz_uint flags) + { + pZip->m_pWrite = mz_zip_file_write_func; + pZip->m_pNeeds_keepalive = NULL; + + if (flags & MZ_ZIP_FLAG_WRITE_ALLOW_READING) + pZip->m_pRead = mz_zip_file_read_func; + + pZip->m_pIO_opaque = pZip; + + if (!mz_zip_writer_init_v2(pZip, 0, flags)) + return MZ_FALSE; + + pZip->m_pState->m_pFile = pFile; + pZip->m_pState->m_file_archive_start_ofs = MZ_FTELL64(pZip->m_pState->m_pFile); + pZip->m_zip_type = MZ_ZIP_TYPE_CFILE; + + return MZ_TRUE; + } +#endif /* #ifndef MINIZ_NO_STDIO */ + + mz_bool mz_zip_writer_init_from_reader_v2(mz_zip_archive *pZip, const char *pFilename, mz_uint flags) + { + mz_zip_internal_state *pState; + + if ((!pZip) || (!pZip->m_pState) || (pZip->m_zip_mode != MZ_ZIP_MODE_READING)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (flags & MZ_ZIP_FLAG_WRITE_ZIP64) + { + /* We don't support converting a non-zip64 file to zip64 - this seems like more trouble than it's worth. (What about the existing 32-bit data descriptors that could follow the compressed data?) */ + if (!pZip->m_pState->m_zip64) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + } + + /* No sense in trying to write to an archive that's already at the support max size */ + if (pZip->m_pState->m_zip64) + { + if (pZip->m_total_files == MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + else + { + if (pZip->m_total_files == MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + + if ((pZip->m_archive_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + MZ_ZIP_LOCAL_DIR_HEADER_SIZE) > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_TOO_LARGE); + } + + pState = pZip->m_pState; + + if (pState->m_pFile) + { +#ifdef MINIZ_NO_STDIO + (void)pFilename; + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); +#else + if (pZip->m_pIO_opaque != pZip) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (pZip->m_zip_type == MZ_ZIP_TYPE_FILE && + !(flags & MZ_ZIP_FLAG_READ_ALLOW_WRITING) ) + { + if (!pFilename) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + /* Archive is being read from stdio and was originally opened only for reading. Try to reopen as writable. */ + if (NULL == (pState->m_pFile = MZ_FREOPEN(pFilename, "r+b", pState->m_pFile))) + { + /* The mz_zip_archive is now in a bogus state because pState->m_pFile is NULL, so just close it. */ + mz_zip_reader_end_internal(pZip, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_OPEN_FAILED); + } + } + + pZip->m_pWrite = mz_zip_file_write_func; + pZip->m_pNeeds_keepalive = NULL; +#endif /* #ifdef MINIZ_NO_STDIO */ + } + else if (pState->m_pMem) + { + /* Archive lives in a memory block. Assume it's from the heap that we can resize using the realloc callback. */ + if (pZip->m_pIO_opaque != pZip) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState->m_mem_capacity = pState->m_mem_size; + pZip->m_pWrite = mz_zip_heap_write_func; + pZip->m_pNeeds_keepalive = NULL; + } + /* Archive is being read via a user provided read function - make sure the user has specified a write function too. */ + else if (!pZip->m_pWrite) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + /* Start writing new files at the archive's current central directory location. */ + /* TODO: We could add a flag that lets the user start writing immediately AFTER the existing central dir - this would be safer. */ + pZip->m_archive_size = pZip->m_central_directory_file_ofs; + pZip->m_central_directory_file_ofs = 0; + + /* Clear the sorted central dir offsets, they aren't useful or maintained now. */ + /* Even though we're now in write mode, files can still be extracted and verified, but file locates will be slow. */ + /* TODO: We could easily maintain the sorted central directory offsets. */ + mz_zip_array_clear(pZip, &pZip->m_pState->m_sorted_central_dir_offsets); + + pZip->m_zip_mode = MZ_ZIP_MODE_WRITING; + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_init_from_reader(mz_zip_archive *pZip, const char *pFilename) + { + return mz_zip_writer_init_from_reader_v2(pZip, pFilename, 0); + } + + /* TODO: pArchive_name is a terrible name here! */ + mz_bool mz_zip_writer_add_mem(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, mz_uint level_and_flags) + { + return mz_zip_writer_add_mem_ex(pZip, pArchive_name, pBuf, buf_size, NULL, 0, level_and_flags, 0, 0); + } + + typedef struct + { + mz_zip_archive *m_pZip; + mz_uint64 m_cur_archive_file_ofs; + mz_uint64 m_comp_size; + } mz_zip_writer_add_state; + + static mz_bool mz_zip_writer_add_put_buf_callback(const void *pBuf, int len, void *pUser) + { + mz_zip_writer_add_state *pState = (mz_zip_writer_add_state *)pUser; + if ((int)pState->m_pZip->m_pWrite(pState->m_pZip->m_pIO_opaque, pState->m_cur_archive_file_ofs, pBuf, len) != len) + return MZ_FALSE; + + pState->m_cur_archive_file_ofs += len; + pState->m_comp_size += len; + return MZ_TRUE; + } + +#define MZ_ZIP64_MAX_LOCAL_EXTRA_FIELD_SIZE (sizeof(mz_uint16) * 2 + sizeof(mz_uint64) * 2) +#define MZ_ZIP64_MAX_CENTRAL_EXTRA_FIELD_SIZE (sizeof(mz_uint16) * 2 + sizeof(mz_uint64) * 3) + static mz_uint32 mz_zip_writer_create_zip64_extra_data(mz_uint8 *pBuf, mz_uint64 *pUncomp_size, mz_uint64 *pComp_size, mz_uint64 *pLocal_header_ofs) + { + mz_uint8 *pDst = pBuf; + mz_uint32 field_size = 0; + + MZ_WRITE_LE16(pDst + 0, MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID); + MZ_WRITE_LE16(pDst + 2, 0); + pDst += sizeof(mz_uint16) * 2; + + if (pUncomp_size) + { + MZ_WRITE_LE64(pDst, *pUncomp_size); + pDst += sizeof(mz_uint64); + field_size += sizeof(mz_uint64); + } + + if (pComp_size) + { + MZ_WRITE_LE64(pDst, *pComp_size); + pDst += sizeof(mz_uint64); + field_size += sizeof(mz_uint64); + } + + if (pLocal_header_ofs) + { + MZ_WRITE_LE64(pDst, *pLocal_header_ofs); + pDst += sizeof(mz_uint64); + field_size += sizeof(mz_uint64); + } + + MZ_WRITE_LE16(pBuf + 2, field_size); + + return (mz_uint32)(pDst - pBuf); + } + + static mz_bool mz_zip_writer_create_local_dir_header(mz_zip_archive *pZip, mz_uint8 *pDst, mz_uint16 filename_size, mz_uint16 extra_size, mz_uint64 uncomp_size, mz_uint64 comp_size, mz_uint32 uncomp_crc32, mz_uint16 method, mz_uint16 bit_flags, mz_uint16 dos_time, mz_uint16 dos_date) + { + (void)pZip; + memset(pDst, 0, MZ_ZIP_LOCAL_DIR_HEADER_SIZE); + MZ_WRITE_LE32(pDst + MZ_ZIP_LDH_SIG_OFS, MZ_ZIP_LOCAL_DIR_HEADER_SIG); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_VERSION_NEEDED_OFS, method ? 20 : 0); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_BIT_FLAG_OFS, bit_flags); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_METHOD_OFS, method); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_FILE_TIME_OFS, dos_time); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_FILE_DATE_OFS, dos_date); + MZ_WRITE_LE32(pDst + MZ_ZIP_LDH_CRC32_OFS, uncomp_crc32); + MZ_WRITE_LE32(pDst + MZ_ZIP_LDH_COMPRESSED_SIZE_OFS, MZ_MIN(comp_size, MZ_UINT32_MAX)); + MZ_WRITE_LE32(pDst + MZ_ZIP_LDH_DECOMPRESSED_SIZE_OFS, MZ_MIN(uncomp_size, MZ_UINT32_MAX)); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_FILENAME_LEN_OFS, filename_size); + MZ_WRITE_LE16(pDst + MZ_ZIP_LDH_EXTRA_LEN_OFS, extra_size); + return MZ_TRUE; + } + + static mz_bool mz_zip_writer_create_central_dir_header(mz_zip_archive *pZip, mz_uint8 *pDst, + mz_uint16 filename_size, mz_uint16 extra_size, mz_uint16 comment_size, + mz_uint64 uncomp_size, mz_uint64 comp_size, mz_uint32 uncomp_crc32, + mz_uint16 method, mz_uint16 bit_flags, mz_uint16 dos_time, mz_uint16 dos_date, + mz_uint64 local_header_ofs, mz_uint32 ext_attributes) + { + (void)pZip; + memset(pDst, 0, MZ_ZIP_CENTRAL_DIR_HEADER_SIZE); + MZ_WRITE_LE32(pDst + MZ_ZIP_CDH_SIG_OFS, MZ_ZIP_CENTRAL_DIR_HEADER_SIG); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_VERSION_NEEDED_OFS, method ? 20 : 0); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_BIT_FLAG_OFS, bit_flags); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_METHOD_OFS, method); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_FILE_TIME_OFS, dos_time); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_FILE_DATE_OFS, dos_date); + MZ_WRITE_LE32(pDst + MZ_ZIP_CDH_CRC32_OFS, uncomp_crc32); + MZ_WRITE_LE32(pDst + MZ_ZIP_CDH_COMPRESSED_SIZE_OFS, MZ_MIN(comp_size, MZ_UINT32_MAX)); + MZ_WRITE_LE32(pDst + MZ_ZIP_CDH_DECOMPRESSED_SIZE_OFS, MZ_MIN(uncomp_size, MZ_UINT32_MAX)); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_FILENAME_LEN_OFS, filename_size); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_EXTRA_LEN_OFS, extra_size); + MZ_WRITE_LE16(pDst + MZ_ZIP_CDH_COMMENT_LEN_OFS, comment_size); + MZ_WRITE_LE32(pDst + MZ_ZIP_CDH_EXTERNAL_ATTR_OFS, ext_attributes); + MZ_WRITE_LE32(pDst + MZ_ZIP_CDH_LOCAL_HEADER_OFS, MZ_MIN(local_header_ofs, MZ_UINT32_MAX)); + return MZ_TRUE; + } + + static mz_bool mz_zip_writer_add_to_central_dir(mz_zip_archive *pZip, const char *pFilename, mz_uint16 filename_size, + const void *pExtra, mz_uint16 extra_size, const void *pComment, mz_uint16 comment_size, + mz_uint64 uncomp_size, mz_uint64 comp_size, mz_uint32 uncomp_crc32, + mz_uint16 method, mz_uint16 bit_flags, mz_uint16 dos_time, mz_uint16 dos_date, + mz_uint64 local_header_ofs, mz_uint32 ext_attributes, + const char *user_extra_data, mz_uint user_extra_data_len) + { + mz_zip_internal_state *pState = pZip->m_pState; + mz_uint32 central_dir_ofs = (mz_uint32)pState->m_central_dir.m_size; + size_t orig_central_dir_size = pState->m_central_dir.m_size; + mz_uint8 central_dir_header[MZ_ZIP_CENTRAL_DIR_HEADER_SIZE]; + + if (!pZip->m_pState->m_zip64) + { + if (local_header_ofs > 0xFFFFFFFF) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_TOO_LARGE); + } + + /* miniz doesn't support central dirs >= MZ_UINT32_MAX bytes yet */ + if (((mz_uint64)pState->m_central_dir.m_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + filename_size + extra_size + user_extra_data_len + comment_size) >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_CDIR_SIZE); + + if (!mz_zip_writer_create_central_dir_header(pZip, central_dir_header, filename_size, (mz_uint16)(extra_size + user_extra_data_len), comment_size, uncomp_size, comp_size, uncomp_crc32, method, bit_flags, dos_time, dos_date, local_header_ofs, ext_attributes)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if ((!mz_zip_array_push_back(pZip, &pState->m_central_dir, central_dir_header, MZ_ZIP_CENTRAL_DIR_HEADER_SIZE)) || + (!mz_zip_array_push_back(pZip, &pState->m_central_dir, pFilename, filename_size)) || + (!mz_zip_array_push_back(pZip, &pState->m_central_dir, pExtra, extra_size)) || + (!mz_zip_array_push_back(pZip, &pState->m_central_dir, user_extra_data, user_extra_data_len)) || + (!mz_zip_array_push_back(pZip, &pState->m_central_dir, pComment, comment_size)) || + (!mz_zip_array_push_back(pZip, &pState->m_central_dir_offsets, ¢ral_dir_ofs, 1))) + { + /* Try to resize the central directory array back into its original state. */ + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + return MZ_TRUE; + } + + static mz_bool mz_zip_writer_validate_archive_name(const char *pArchive_name) + { + /* Basic ZIP archive filename validity checks: Valid filenames cannot start with a forward slash, cannot contain a drive letter, and cannot use DOS-style backward slashes. */ + if (*pArchive_name == '/') + return MZ_FALSE; + + /* Making sure the name does not contain drive letters or DOS style backward slashes is the responsibility of the program using miniz*/ + + return MZ_TRUE; + } + + static mz_uint mz_zip_writer_compute_padding_needed_for_file_alignment(mz_zip_archive *pZip) + { + mz_uint32 n; + if (!pZip->m_file_offset_alignment) + return 0; + n = (mz_uint32)(pZip->m_archive_size & (pZip->m_file_offset_alignment - 1)); + return (mz_uint)((pZip->m_file_offset_alignment - n) & (pZip->m_file_offset_alignment - 1)); + } + + static mz_bool mz_zip_writer_write_zeros(mz_zip_archive *pZip, mz_uint64 cur_file_ofs, mz_uint32 n) + { + char buf[4096]; + memset(buf, 0, MZ_MIN(sizeof(buf), n)); + while (n) + { + mz_uint32 s = MZ_MIN(sizeof(buf), n); + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_file_ofs, buf, s) != s) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_file_ofs += s; + n -= s; + } + return MZ_TRUE; + } + + mz_bool mz_zip_writer_add_mem_ex(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, + mz_uint64 uncomp_size, mz_uint32 uncomp_crc32) + { + return mz_zip_writer_add_mem_ex_v2(pZip, pArchive_name, pBuf, buf_size, pComment, comment_size, level_and_flags, uncomp_size, uncomp_crc32, NULL, NULL, 0, NULL, 0); + } + + mz_bool mz_zip_writer_add_mem_ex_v2(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, + mz_uint level_and_flags, mz_uint64 uncomp_size, mz_uint32 uncomp_crc32, MZ_TIME_T *last_modified, + const char *user_extra_data, mz_uint user_extra_data_len, const char *user_extra_data_central, mz_uint user_extra_data_central_len) + { + mz_uint16 method = 0, dos_time = 0, dos_date = 0; + mz_uint level, ext_attributes = 0, num_alignment_padding_bytes; + mz_uint64 local_dir_header_ofs = pZip->m_archive_size, cur_archive_file_ofs = pZip->m_archive_size, comp_size = 0; + size_t archive_name_size; + mz_uint8 local_dir_header[MZ_ZIP_LOCAL_DIR_HEADER_SIZE]; + tdefl_compressor *pComp = NULL; + mz_bool store_data_uncompressed; + mz_zip_internal_state *pState; + mz_uint8 *pExtra_data = NULL; + mz_uint32 extra_size = 0; + mz_uint8 extra_data[MZ_ZIP64_MAX_CENTRAL_EXTRA_FIELD_SIZE]; + mz_uint16 bit_flags = 0; + + if ((int)level_and_flags < 0) + level_and_flags = MZ_DEFAULT_LEVEL; + + if (uncomp_size || (buf_size && !(level_and_flags & MZ_ZIP_FLAG_COMPRESSED_DATA))) + bit_flags |= MZ_ZIP_LDH_BIT_FLAG_HAS_LOCATOR; + + if (!(level_and_flags & MZ_ZIP_FLAG_ASCII_FILENAME)) + bit_flags |= MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_UTF8; + + level = level_and_flags & 0xF; + store_data_uncompressed = ((!level) || (level_and_flags & MZ_ZIP_FLAG_COMPRESSED_DATA)); + + if ((!pZip) || (!pZip->m_pState) || (pZip->m_zip_mode != MZ_ZIP_MODE_WRITING) || ((buf_size) && (!pBuf)) || (!pArchive_name) || ((comment_size) && (!pComment)) || (level > MZ_UBER_COMPRESSION)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState = pZip->m_pState; + + if (pState->m_zip64) + { + if (pZip->m_total_files == MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + else + { + if (pZip->m_total_files == MZ_UINT16_MAX) + { + pState->m_zip64 = MZ_TRUE; + /*return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); */ + } + if (((mz_uint64)buf_size > 0xFFFFFFFF) || (uncomp_size > 0xFFFFFFFF)) + { + pState->m_zip64 = MZ_TRUE; + /*return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); */ + } + } + + if ((!(level_and_flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) && (uncomp_size)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (!mz_zip_writer_validate_archive_name(pArchive_name)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_FILENAME); + +#ifndef MINIZ_NO_TIME + if (last_modified != NULL) + { + mz_zip_time_t_to_dos_time(*last_modified, &dos_time, &dos_date); + } + else + { + MZ_TIME_T cur_time; + time(&cur_time); + mz_zip_time_t_to_dos_time(cur_time, &dos_time, &dos_date); + } +#else + (void)last_modified; +#endif /* #ifndef MINIZ_NO_TIME */ + + if (!(level_and_flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) + { + uncomp_crc32 = (mz_uint32)mz_crc32(MZ_CRC32_INIT, (const mz_uint8 *)pBuf, buf_size); + uncomp_size = buf_size; + if (uncomp_size <= 3) + { + level = 0; + store_data_uncompressed = MZ_TRUE; + } + } + + archive_name_size = strlen(pArchive_name); + if (archive_name_size > MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_FILENAME); + + num_alignment_padding_bytes = mz_zip_writer_compute_padding_needed_for_file_alignment(pZip); + + /* miniz doesn't support central dirs >= MZ_UINT32_MAX bytes yet */ + if (((mz_uint64)pState->m_central_dir.m_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + archive_name_size + MZ_ZIP64_MAX_CENTRAL_EXTRA_FIELD_SIZE + comment_size) >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_CDIR_SIZE); + + if (!pState->m_zip64) + { + /* Bail early if the archive would obviously become too large */ + if ((pZip->m_archive_size + num_alignment_padding_bytes + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + archive_name_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + archive_name_size + comment_size + user_extra_data_len + + pState->m_central_dir.m_size + MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE + user_extra_data_central_len + MZ_ZIP_DATA_DESCRIPTER_SIZE32) > 0xFFFFFFFF) + { + pState->m_zip64 = MZ_TRUE; + /*return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); */ + } + } + + if ((archive_name_size) && (pArchive_name[archive_name_size - 1] == '/')) + { + /* Set DOS Subdirectory attribute bit. */ + ext_attributes |= MZ_ZIP_DOS_DIR_ATTRIBUTE_BITFLAG; + + /* Subdirectories cannot contain data. */ + if ((buf_size) || (uncomp_size)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + } + + /* Try to do any allocations before writing to the archive, so if an allocation fails the file remains unmodified. (A good idea if we're doing an in-place modification.) */ + if ((!mz_zip_array_ensure_room(pZip, &pState->m_central_dir, MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + archive_name_size + comment_size + (pState->m_zip64 ? MZ_ZIP64_MAX_CENTRAL_EXTRA_FIELD_SIZE : 0))) || (!mz_zip_array_ensure_room(pZip, &pState->m_central_dir_offsets, 1))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + if ((!store_data_uncompressed) && (buf_size)) + { + if (NULL == (pComp = (tdefl_compressor *)pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, sizeof(tdefl_compressor)))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (!mz_zip_writer_write_zeros(pZip, cur_archive_file_ofs, num_alignment_padding_bytes)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + return MZ_FALSE; + } + + local_dir_header_ofs += num_alignment_padding_bytes; + if (pZip->m_file_offset_alignment) + { + MZ_ASSERT((local_dir_header_ofs & (pZip->m_file_offset_alignment - 1)) == 0); + } + cur_archive_file_ofs += num_alignment_padding_bytes; + + MZ_CLEAR_ARR(local_dir_header); + + if (!store_data_uncompressed || (level_and_flags & MZ_ZIP_FLAG_COMPRESSED_DATA)) + { + method = MZ_DEFLATED; + } + + if (pState->m_zip64) + { + if (uncomp_size >= MZ_UINT32_MAX || local_dir_header_ofs >= MZ_UINT32_MAX) + { + pExtra_data = extra_data; + extra_size = mz_zip_writer_create_zip64_extra_data(extra_data, (uncomp_size >= MZ_UINT32_MAX) ? &uncomp_size : NULL, + (uncomp_size >= MZ_UINT32_MAX) ? &comp_size : NULL, (local_dir_header_ofs >= MZ_UINT32_MAX) ? &local_dir_header_ofs : NULL); + } + + if (!mz_zip_writer_create_local_dir_header(pZip, local_dir_header, (mz_uint16)archive_name_size, (mz_uint16)(extra_size + user_extra_data_len), 0, 0, 0, method, bit_flags, dos_time, dos_date)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, local_dir_header_ofs, local_dir_header, sizeof(local_dir_header)) != sizeof(local_dir_header)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += sizeof(local_dir_header); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, pArchive_name, archive_name_size) != archive_name_size) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + cur_archive_file_ofs += archive_name_size; + + if (pExtra_data != NULL) + { + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, extra_data, extra_size) != extra_size) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += extra_size; + } + } + else + { + if ((comp_size > MZ_UINT32_MAX) || (cur_archive_file_ofs > MZ_UINT32_MAX)) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + if (!mz_zip_writer_create_local_dir_header(pZip, local_dir_header, (mz_uint16)archive_name_size, (mz_uint16)user_extra_data_len, 0, 0, 0, method, bit_flags, dos_time, dos_date)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, local_dir_header_ofs, local_dir_header, sizeof(local_dir_header)) != sizeof(local_dir_header)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += sizeof(local_dir_header); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, pArchive_name, archive_name_size) != archive_name_size) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + cur_archive_file_ofs += archive_name_size; + } + + if (user_extra_data_len > 0) + { + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, user_extra_data, user_extra_data_len) != user_extra_data_len) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += user_extra_data_len; + } + + if (store_data_uncompressed) + { + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, pBuf, buf_size) != buf_size) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + + cur_archive_file_ofs += buf_size; + comp_size = buf_size; + } + else if (buf_size) + { + mz_zip_writer_add_state state; + + state.m_pZip = pZip; + state.m_cur_archive_file_ofs = cur_archive_file_ofs; + state.m_comp_size = 0; + + if ((tdefl_init(pComp, mz_zip_writer_add_put_buf_callback, &state, tdefl_create_comp_flags_from_zip_params(level, -15, MZ_DEFAULT_STRATEGY)) != TDEFL_STATUS_OKAY) || + (tdefl_compress_buffer(pComp, pBuf, buf_size, TDEFL_FINISH) != TDEFL_STATUS_DONE)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + return mz_zip_set_error(pZip, MZ_ZIP_COMPRESSION_FAILED); + } + + comp_size = state.m_comp_size; + cur_archive_file_ofs = state.m_cur_archive_file_ofs; + } + + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + pComp = NULL; + + if (uncomp_size) + { + mz_uint8 local_dir_footer[MZ_ZIP_DATA_DESCRIPTER_SIZE64]; + mz_uint32 local_dir_footer_size = MZ_ZIP_DATA_DESCRIPTER_SIZE32; + + MZ_ASSERT(bit_flags & MZ_ZIP_LDH_BIT_FLAG_HAS_LOCATOR); + + MZ_WRITE_LE32(local_dir_footer + 0, MZ_ZIP_DATA_DESCRIPTOR_ID); + MZ_WRITE_LE32(local_dir_footer + 4, uncomp_crc32); + if (pExtra_data == NULL) + { + if (comp_size > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + + MZ_WRITE_LE32(local_dir_footer + 8, comp_size); + MZ_WRITE_LE32(local_dir_footer + 12, uncomp_size); + } + else + { + MZ_WRITE_LE64(local_dir_footer + 8, comp_size); + MZ_WRITE_LE64(local_dir_footer + 16, uncomp_size); + local_dir_footer_size = MZ_ZIP_DATA_DESCRIPTER_SIZE64; + } + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, local_dir_footer, local_dir_footer_size) != local_dir_footer_size) + return MZ_FALSE; + + cur_archive_file_ofs += local_dir_footer_size; + } + + if (pExtra_data != NULL) + { + extra_size = mz_zip_writer_create_zip64_extra_data(extra_data, (uncomp_size >= MZ_UINT32_MAX) ? &uncomp_size : NULL, + (uncomp_size >= MZ_UINT32_MAX) ? &comp_size : NULL, (local_dir_header_ofs >= MZ_UINT32_MAX) ? &local_dir_header_ofs : NULL); + } + + if (!mz_zip_writer_add_to_central_dir(pZip, pArchive_name, (mz_uint16)archive_name_size, pExtra_data, (mz_uint16)extra_size, pComment, + comment_size, uncomp_size, comp_size, uncomp_crc32, method, bit_flags, dos_time, dos_date, local_dir_header_ofs, ext_attributes, + user_extra_data_central, user_extra_data_central_len)) + return MZ_FALSE; + + pZip->m_total_files++; + pZip->m_archive_size = cur_archive_file_ofs; + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_add_read_buf_callback(mz_zip_archive *pZip, const char *pArchive_name, mz_file_read_func read_callback, void *callback_opaque, mz_uint64 max_size, const MZ_TIME_T *pFile_time, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, + const char *user_extra_data, mz_uint user_extra_data_len, const char *user_extra_data_central, mz_uint user_extra_data_central_len) + { + mz_uint16 gen_flags; + mz_uint uncomp_crc32 = MZ_CRC32_INIT, level, num_alignment_padding_bytes; + mz_uint16 method = 0, dos_time = 0, dos_date = 0, ext_attributes = 0; + mz_uint64 local_dir_header_ofs, cur_archive_file_ofs = pZip->m_archive_size, uncomp_size = 0, comp_size = 0; + size_t archive_name_size; + mz_uint8 local_dir_header[MZ_ZIP_LOCAL_DIR_HEADER_SIZE]; + mz_uint8 *pExtra_data = NULL; + mz_uint32 extra_size = 0; + mz_uint8 extra_data[MZ_ZIP64_MAX_CENTRAL_EXTRA_FIELD_SIZE]; + mz_zip_internal_state *pState; + mz_uint64 file_ofs = 0, cur_archive_header_file_ofs; + + if ((int)level_and_flags < 0) + level_and_flags = MZ_DEFAULT_LEVEL; + level = level_and_flags & 0xF; + + gen_flags = (level_and_flags & MZ_ZIP_FLAG_WRITE_HEADER_SET_SIZE) ? 0 : MZ_ZIP_LDH_BIT_FLAG_HAS_LOCATOR; + + if (!(level_and_flags & MZ_ZIP_FLAG_ASCII_FILENAME)) + gen_flags |= MZ_ZIP_GENERAL_PURPOSE_BIT_FLAG_UTF8; + + /* Sanity checks */ + if ((!pZip) || (!pZip->m_pState) || (pZip->m_zip_mode != MZ_ZIP_MODE_WRITING) || (!pArchive_name) || ((comment_size) && (!pComment)) || (level > MZ_UBER_COMPRESSION)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState = pZip->m_pState; + + if ((!pState->m_zip64) && (max_size > MZ_UINT32_MAX)) + { + /* Source file is too large for non-zip64 */ + /*return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); */ + pState->m_zip64 = MZ_TRUE; + } + + /* We could support this, but why? */ + if (level_and_flags & MZ_ZIP_FLAG_COMPRESSED_DATA) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (!mz_zip_writer_validate_archive_name(pArchive_name)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_FILENAME); + + if (pState->m_zip64) + { + if (pZip->m_total_files == MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + else + { + if (pZip->m_total_files == MZ_UINT16_MAX) + { + pState->m_zip64 = MZ_TRUE; + /*return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); */ + } + } + + archive_name_size = strlen(pArchive_name); + if (archive_name_size > MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_FILENAME); + + num_alignment_padding_bytes = mz_zip_writer_compute_padding_needed_for_file_alignment(pZip); + + /* miniz doesn't support central dirs >= MZ_UINT32_MAX bytes yet */ + if (((mz_uint64)pState->m_central_dir.m_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + archive_name_size + MZ_ZIP64_MAX_CENTRAL_EXTRA_FIELD_SIZE + comment_size) >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_CDIR_SIZE); + + if (!pState->m_zip64) + { + /* Bail early if the archive would obviously become too large */ + if ((pZip->m_archive_size + num_alignment_padding_bytes + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + archive_name_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + archive_name_size + comment_size + user_extra_data_len + pState->m_central_dir.m_size + MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE + 1024 + MZ_ZIP_DATA_DESCRIPTER_SIZE32 + user_extra_data_central_len) > 0xFFFFFFFF) + { + pState->m_zip64 = MZ_TRUE; + /*return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); */ + } + } + +#ifndef MINIZ_NO_TIME + if (pFile_time) + { + mz_zip_time_t_to_dos_time(*pFile_time, &dos_time, &dos_date); + } +#else + (void)pFile_time; +#endif + + if (max_size <= 3) + level = 0; + + if (!mz_zip_writer_write_zeros(pZip, cur_archive_file_ofs, num_alignment_padding_bytes)) + { + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + + cur_archive_file_ofs += num_alignment_padding_bytes; + local_dir_header_ofs = cur_archive_file_ofs; + + if (pZip->m_file_offset_alignment) + { + MZ_ASSERT((cur_archive_file_ofs & (pZip->m_file_offset_alignment - 1)) == 0); + } + + if (max_size && level) + { + method = MZ_DEFLATED; + } + + MZ_CLEAR_ARR(local_dir_header); + if (pState->m_zip64) + { + if (max_size >= MZ_UINT32_MAX || local_dir_header_ofs >= MZ_UINT32_MAX) + { + pExtra_data = extra_data; + if (level_and_flags & MZ_ZIP_FLAG_WRITE_HEADER_SET_SIZE) + extra_size = mz_zip_writer_create_zip64_extra_data(extra_data, (max_size >= MZ_UINT32_MAX) ? &uncomp_size : NULL, + (max_size >= MZ_UINT32_MAX) ? &comp_size : NULL, + (local_dir_header_ofs >= MZ_UINT32_MAX) ? &local_dir_header_ofs : NULL); + else + extra_size = mz_zip_writer_create_zip64_extra_data(extra_data, NULL, + NULL, + (local_dir_header_ofs >= MZ_UINT32_MAX) ? &local_dir_header_ofs : NULL); + } + + if (!mz_zip_writer_create_local_dir_header(pZip, local_dir_header, (mz_uint16)archive_name_size, (mz_uint16)(extra_size + user_extra_data_len), 0, 0, 0, method, gen_flags, dos_time, dos_date)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, local_dir_header, sizeof(local_dir_header)) != sizeof(local_dir_header)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += sizeof(local_dir_header); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, pArchive_name, archive_name_size) != archive_name_size) + { + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + + cur_archive_file_ofs += archive_name_size; + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, extra_data, extra_size) != extra_size) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += extra_size; + } + else + { + if ((comp_size > MZ_UINT32_MAX) || (cur_archive_file_ofs > MZ_UINT32_MAX)) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + if (!mz_zip_writer_create_local_dir_header(pZip, local_dir_header, (mz_uint16)archive_name_size, (mz_uint16)user_extra_data_len, 0, 0, 0, method, gen_flags, dos_time, dos_date)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, local_dir_header, sizeof(local_dir_header)) != sizeof(local_dir_header)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += sizeof(local_dir_header); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, pArchive_name, archive_name_size) != archive_name_size) + { + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + + cur_archive_file_ofs += archive_name_size; + } + + if (user_extra_data_len > 0) + { + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, user_extra_data, user_extra_data_len) != user_extra_data_len) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_file_ofs += user_extra_data_len; + } + + if (max_size) + { + void *pRead_buf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, MZ_ZIP_MAX_IO_BUF_SIZE); + if (!pRead_buf) + { + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (!level) + { + while (1) + { + size_t n = read_callback(callback_opaque, file_ofs, pRead_buf, MZ_ZIP_MAX_IO_BUF_SIZE); + if (n == 0) + break; + + if ((n > MZ_ZIP_MAX_IO_BUF_SIZE) || (file_ofs + n > max_size)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + } + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, pRead_buf, n) != n) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + file_ofs += n; + uncomp_crc32 = (mz_uint32)mz_crc32(uncomp_crc32, (const mz_uint8 *)pRead_buf, n); + cur_archive_file_ofs += n; + } + uncomp_size = file_ofs; + comp_size = uncomp_size; + } + else + { + mz_bool result = MZ_FALSE; + mz_zip_writer_add_state state; + tdefl_compressor *pComp = (tdefl_compressor *)pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, sizeof(tdefl_compressor)); + if (!pComp) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + state.m_pZip = pZip; + state.m_cur_archive_file_ofs = cur_archive_file_ofs; + state.m_comp_size = 0; + + if (tdefl_init(pComp, mz_zip_writer_add_put_buf_callback, &state, tdefl_create_comp_flags_from_zip_params(level, -15, MZ_DEFAULT_STRATEGY)) != TDEFL_STATUS_OKAY) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + } + + for (;;) + { + tdefl_status status; + tdefl_flush flush = TDEFL_NO_FLUSH; + + size_t n = read_callback(callback_opaque, file_ofs, pRead_buf, MZ_ZIP_MAX_IO_BUF_SIZE); + if ((n > MZ_ZIP_MAX_IO_BUF_SIZE) || (file_ofs + n > max_size)) + { + mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + break; + } + + file_ofs += n; + uncomp_crc32 = (mz_uint32)mz_crc32(uncomp_crc32, (const mz_uint8 *)pRead_buf, n); + + if (pZip->m_pNeeds_keepalive != NULL && pZip->m_pNeeds_keepalive(pZip->m_pIO_opaque)) + flush = TDEFL_FULL_FLUSH; + + if (n == 0) + flush = TDEFL_FINISH; + + status = tdefl_compress_buffer(pComp, pRead_buf, n, flush); + if (status == TDEFL_STATUS_DONE) + { + result = MZ_TRUE; + break; + } + else if (status != TDEFL_STATUS_OKAY) + { + mz_zip_set_error(pZip, MZ_ZIP_COMPRESSION_FAILED); + break; + } + } + + pZip->m_pFree(pZip->m_pAlloc_opaque, pComp); + + if (!result) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + return MZ_FALSE; + } + + uncomp_size = file_ofs; + comp_size = state.m_comp_size; + cur_archive_file_ofs = state.m_cur_archive_file_ofs; + } + + pZip->m_pFree(pZip->m_pAlloc_opaque, pRead_buf); + } + + if (!(level_and_flags & MZ_ZIP_FLAG_WRITE_HEADER_SET_SIZE)) + { + mz_uint8 local_dir_footer[MZ_ZIP_DATA_DESCRIPTER_SIZE64]; + mz_uint32 local_dir_footer_size = MZ_ZIP_DATA_DESCRIPTER_SIZE32; + + MZ_WRITE_LE32(local_dir_footer + 0, MZ_ZIP_DATA_DESCRIPTOR_ID); + MZ_WRITE_LE32(local_dir_footer + 4, uncomp_crc32); + if (pExtra_data == NULL) + { + if (comp_size > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + + MZ_WRITE_LE32(local_dir_footer + 8, comp_size); + MZ_WRITE_LE32(local_dir_footer + 12, uncomp_size); + } + else + { + MZ_WRITE_LE64(local_dir_footer + 8, comp_size); + MZ_WRITE_LE64(local_dir_footer + 16, uncomp_size); + local_dir_footer_size = MZ_ZIP_DATA_DESCRIPTER_SIZE64; + } + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_file_ofs, local_dir_footer, local_dir_footer_size) != local_dir_footer_size) + return MZ_FALSE; + + cur_archive_file_ofs += local_dir_footer_size; + } + + if (level_and_flags & MZ_ZIP_FLAG_WRITE_HEADER_SET_SIZE) + { + if (pExtra_data != NULL) + { + extra_size = mz_zip_writer_create_zip64_extra_data(extra_data, (max_size >= MZ_UINT32_MAX) ? &uncomp_size : NULL, + (max_size >= MZ_UINT32_MAX) ? &comp_size : NULL, (local_dir_header_ofs >= MZ_UINT32_MAX) ? &local_dir_header_ofs : NULL); + } + + if (!mz_zip_writer_create_local_dir_header(pZip, local_dir_header, + (mz_uint16)archive_name_size, (mz_uint16)(extra_size + user_extra_data_len), + (max_size >= MZ_UINT32_MAX) ? MZ_UINT32_MAX : uncomp_size, + (max_size >= MZ_UINT32_MAX) ? MZ_UINT32_MAX : comp_size, + uncomp_crc32, method, gen_flags, dos_time, dos_date)) + return mz_zip_set_error(pZip, MZ_ZIP_INTERNAL_ERROR); + + cur_archive_header_file_ofs = local_dir_header_ofs; + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_header_file_ofs, local_dir_header, sizeof(local_dir_header)) != sizeof(local_dir_header)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + if (pExtra_data != NULL) + { + cur_archive_header_file_ofs += sizeof(local_dir_header); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_header_file_ofs, pArchive_name, archive_name_size) != archive_name_size) + { + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + + cur_archive_header_file_ofs += archive_name_size; + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_archive_header_file_ofs, extra_data, extra_size) != extra_size) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_archive_header_file_ofs += extra_size; + } + } + + if (pExtra_data != NULL) + { + extra_size = mz_zip_writer_create_zip64_extra_data(extra_data, (uncomp_size >= MZ_UINT32_MAX) ? &uncomp_size : NULL, + (uncomp_size >= MZ_UINT32_MAX) ? &comp_size : NULL, (local_dir_header_ofs >= MZ_UINT32_MAX) ? &local_dir_header_ofs : NULL); + } + + if (!mz_zip_writer_add_to_central_dir(pZip, pArchive_name, (mz_uint16)archive_name_size, pExtra_data, (mz_uint16)extra_size, pComment, comment_size, + uncomp_size, comp_size, uncomp_crc32, method, gen_flags, dos_time, dos_date, local_dir_header_ofs, ext_attributes, + user_extra_data_central, user_extra_data_central_len)) + return MZ_FALSE; + + pZip->m_total_files++; + pZip->m_archive_size = cur_archive_file_ofs; + + return MZ_TRUE; + } + +#ifndef MINIZ_NO_STDIO + + static size_t mz_file_read_func_stdio(void *pOpaque, mz_uint64 file_ofs, void *pBuf, size_t n) + { + MZ_FILE *pSrc_file = (MZ_FILE *)pOpaque; + mz_int64 cur_ofs = MZ_FTELL64(pSrc_file); + + if (((mz_int64)file_ofs < 0) || (((cur_ofs != (mz_int64)file_ofs)) && (MZ_FSEEK64(pSrc_file, (mz_int64)file_ofs, SEEK_SET)))) + return 0; + + return MZ_FREAD(pBuf, 1, n, pSrc_file); + } + + mz_bool mz_zip_writer_add_cfile(mz_zip_archive *pZip, const char *pArchive_name, MZ_FILE *pSrc_file, mz_uint64 max_size, const MZ_TIME_T *pFile_time, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, + const char *user_extra_data, mz_uint user_extra_data_len, const char *user_extra_data_central, mz_uint user_extra_data_central_len) + { + return mz_zip_writer_add_read_buf_callback(pZip, pArchive_name, mz_file_read_func_stdio, pSrc_file, max_size, pFile_time, pComment, comment_size, level_and_flags, + user_extra_data, user_extra_data_len, user_extra_data_central, user_extra_data_central_len); + } + + mz_bool mz_zip_writer_add_file(mz_zip_archive *pZip, const char *pArchive_name, const char *pSrc_filename, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags) + { + MZ_FILE *pSrc_file = NULL; + mz_uint64 uncomp_size = 0; + MZ_TIME_T file_modified_time; + MZ_TIME_T *pFile_time = NULL; + mz_bool status; + + memset(&file_modified_time, 0, sizeof(file_modified_time)); + +#if !defined(MINIZ_NO_TIME) && !defined(MINIZ_NO_STDIO) + pFile_time = &file_modified_time; + if (!mz_zip_get_file_modified_time(pSrc_filename, &file_modified_time)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_STAT_FAILED); +#endif + + pSrc_file = MZ_FOPEN(pSrc_filename, "rb"); + if (!pSrc_file) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_OPEN_FAILED); + + MZ_FSEEK64(pSrc_file, 0, SEEK_END); + uncomp_size = MZ_FTELL64(pSrc_file); + MZ_FSEEK64(pSrc_file, 0, SEEK_SET); + + status = mz_zip_writer_add_cfile(pZip, pArchive_name, pSrc_file, uncomp_size, pFile_time, pComment, comment_size, level_and_flags, NULL, 0, NULL, 0); + + MZ_FCLOSE(pSrc_file); + + return status; + } +#endif /* #ifndef MINIZ_NO_STDIO */ + + static mz_bool mz_zip_writer_update_zip64_extension_block(mz_zip_array *pNew_ext, mz_zip_archive *pZip, const mz_uint8 *pExt, mz_uint32 ext_len, mz_uint64 *pComp_size, mz_uint64 *pUncomp_size, mz_uint64 *pLocal_header_ofs, mz_uint32 *pDisk_start) + { + /* + 64 should be enough for any new zip64 data */ + if (!mz_zip_array_reserve(pZip, pNew_ext, ext_len + 64, MZ_FALSE)) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + mz_zip_array_resize(pZip, pNew_ext, 0, MZ_FALSE); + + if ((pUncomp_size) || (pComp_size) || (pLocal_header_ofs) || (pDisk_start)) + { + mz_uint8 new_ext_block[64]; + mz_uint8 *pDst = new_ext_block; + mz_write_le16(pDst, MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID); + mz_write_le16(pDst + sizeof(mz_uint16), 0); + pDst += sizeof(mz_uint16) * 2; + + if (pUncomp_size) + { + mz_write_le64(pDst, *pUncomp_size); + pDst += sizeof(mz_uint64); + } + + if (pComp_size) + { + mz_write_le64(pDst, *pComp_size); + pDst += sizeof(mz_uint64); + } + + if (pLocal_header_ofs) + { + mz_write_le64(pDst, *pLocal_header_ofs); + pDst += sizeof(mz_uint64); + } + + if (pDisk_start) + { + mz_write_le32(pDst, *pDisk_start); + pDst += sizeof(mz_uint32); + } + + mz_write_le16(new_ext_block + sizeof(mz_uint16), (mz_uint16)((pDst - new_ext_block) - sizeof(mz_uint16) * 2)); + + if (!mz_zip_array_push_back(pZip, pNew_ext, new_ext_block, pDst - new_ext_block)) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if ((pExt) && (ext_len)) + { + mz_uint32 extra_size_remaining = ext_len; + const mz_uint8 *pExtra_data = pExt; + + do + { + mz_uint32 field_id, field_data_size, field_total_size; + + if (extra_size_remaining < (sizeof(mz_uint16) * 2)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + field_id = MZ_READ_LE16(pExtra_data); + field_data_size = MZ_READ_LE16(pExtra_data + sizeof(mz_uint16)); + field_total_size = field_data_size + sizeof(mz_uint16) * 2; + + if (field_total_size > extra_size_remaining) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + if (field_id != MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID) + { + if (!mz_zip_array_push_back(pZip, pNew_ext, pExtra_data, field_total_size)) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + pExtra_data += field_total_size; + extra_size_remaining -= field_total_size; + } while (extra_size_remaining); + } + + return MZ_TRUE; + } + + /* TODO: This func is now pretty freakin complex due to zip64, split it up? */ + mz_bool mz_zip_writer_add_from_zip_reader(mz_zip_archive *pZip, mz_zip_archive *pSource_zip, mz_uint src_file_index) + { + mz_uint n, bit_flags, num_alignment_padding_bytes, src_central_dir_following_data_size; + mz_uint64 src_archive_bytes_remaining, local_dir_header_ofs; + mz_uint64 cur_src_file_ofs, cur_dst_file_ofs; + mz_uint32 local_header_u32[(MZ_ZIP_LOCAL_DIR_HEADER_SIZE + sizeof(mz_uint32) - 1) / sizeof(mz_uint32)]; + mz_uint8 *pLocal_header = (mz_uint8 *)local_header_u32; + mz_uint8 new_central_header[MZ_ZIP_CENTRAL_DIR_HEADER_SIZE]; + size_t orig_central_dir_size; + mz_zip_internal_state *pState; + void *pBuf; + const mz_uint8 *pSrc_central_header; + mz_zip_archive_file_stat src_file_stat; + mz_uint32 src_filename_len, src_comment_len, src_ext_len; + mz_uint32 local_header_filename_size, local_header_extra_len; + mz_uint64 local_header_comp_size, local_header_uncomp_size; + mz_bool found_zip64_ext_data_in_ldir = MZ_FALSE; + + /* Sanity checks */ + if ((!pZip) || (!pZip->m_pState) || (pZip->m_zip_mode != MZ_ZIP_MODE_WRITING) || (!pSource_zip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState = pZip->m_pState; + + /* Don't support copying files from zip64 archives to non-zip64, even though in some cases this is possible */ + if ((pSource_zip->m_pState->m_zip64) && (!pZip->m_pState->m_zip64)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + /* Get pointer to the source central dir header and crack it */ + if (NULL == (pSrc_central_header = mz_zip_get_cdh(pSource_zip, src_file_index))) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (MZ_READ_LE32(pSrc_central_header + MZ_ZIP_CDH_SIG_OFS) != MZ_ZIP_CENTRAL_DIR_HEADER_SIG) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + src_filename_len = MZ_READ_LE16(pSrc_central_header + MZ_ZIP_CDH_FILENAME_LEN_OFS); + src_comment_len = MZ_READ_LE16(pSrc_central_header + MZ_ZIP_CDH_COMMENT_LEN_OFS); + src_ext_len = MZ_READ_LE16(pSrc_central_header + MZ_ZIP_CDH_EXTRA_LEN_OFS); + src_central_dir_following_data_size = src_filename_len + src_ext_len + src_comment_len; + + /* TODO: We don't support central dir's >= MZ_UINT32_MAX bytes right now (+32 fudge factor in case we need to add more extra data) */ + if ((pState->m_central_dir.m_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + src_central_dir_following_data_size + 32) >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_CDIR_SIZE); + + num_alignment_padding_bytes = mz_zip_writer_compute_padding_needed_for_file_alignment(pZip); + + if (!pState->m_zip64) + { + if (pZip->m_total_files == MZ_UINT16_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + else + { + /* TODO: Our zip64 support still has some 32-bit limits that may not be worth fixing. */ + if (pZip->m_total_files == MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + + if (!mz_zip_file_stat_internal(pSource_zip, src_file_index, pSrc_central_header, &src_file_stat, NULL)) + return MZ_FALSE; + + cur_src_file_ofs = src_file_stat.m_local_header_ofs; + cur_dst_file_ofs = pZip->m_archive_size; + + /* Read the source archive's local dir header */ + if (pSource_zip->m_pRead(pSource_zip->m_pIO_opaque, cur_src_file_ofs, pLocal_header, MZ_ZIP_LOCAL_DIR_HEADER_SIZE) != MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + + if (MZ_READ_LE32(pLocal_header) != MZ_ZIP_LOCAL_DIR_HEADER_SIG) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + + cur_src_file_ofs += MZ_ZIP_LOCAL_DIR_HEADER_SIZE; + + /* Compute the total size we need to copy (filename+extra data+compressed data) */ + local_header_filename_size = MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_FILENAME_LEN_OFS); + local_header_extra_len = MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_EXTRA_LEN_OFS); + local_header_comp_size = MZ_READ_LE32(pLocal_header + MZ_ZIP_LDH_COMPRESSED_SIZE_OFS); + local_header_uncomp_size = MZ_READ_LE32(pLocal_header + MZ_ZIP_LDH_DECOMPRESSED_SIZE_OFS); + src_archive_bytes_remaining = src_file_stat.m_comp_size + local_header_filename_size + local_header_extra_len; + + /* Try to find a zip64 extended information field */ + if ((local_header_extra_len) && ((local_header_comp_size == MZ_UINT32_MAX) || (local_header_uncomp_size == MZ_UINT32_MAX))) + { + mz_zip_array file_data_array; + const mz_uint8 *pExtra_data; + mz_uint32 extra_size_remaining = local_header_extra_len; + + mz_zip_array_init(&file_data_array, 1); + if (!mz_zip_array_resize(pZip, &file_data_array, local_header_extra_len, MZ_FALSE)) + { + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (pSource_zip->m_pRead(pSource_zip->m_pIO_opaque, src_file_stat.m_local_header_ofs + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + local_header_filename_size, file_data_array.m_p, local_header_extra_len) != local_header_extra_len) + { + mz_zip_array_clear(pZip, &file_data_array); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + } + + pExtra_data = (const mz_uint8 *)file_data_array.m_p; + + do + { + mz_uint32 field_id, field_data_size, field_total_size; + + if (extra_size_remaining < (sizeof(mz_uint16) * 2)) + { + mz_zip_array_clear(pZip, &file_data_array); + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + field_id = MZ_READ_LE16(pExtra_data); + field_data_size = MZ_READ_LE16(pExtra_data + sizeof(mz_uint16)); + field_total_size = field_data_size + sizeof(mz_uint16) * 2; + + if (field_total_size > extra_size_remaining) + { + mz_zip_array_clear(pZip, &file_data_array); + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + if (field_id == MZ_ZIP64_EXTENDED_INFORMATION_FIELD_HEADER_ID) + { + const mz_uint8 *pSrc_field_data = pExtra_data + sizeof(mz_uint32); + + if (field_data_size < sizeof(mz_uint64) * 2) + { + mz_zip_array_clear(pZip, &file_data_array); + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_HEADER_OR_CORRUPTED); + } + + local_header_uncomp_size = MZ_READ_LE64(pSrc_field_data); + local_header_comp_size = MZ_READ_LE64(pSrc_field_data + sizeof(mz_uint64)); /* may be 0 if there's a descriptor */ + + found_zip64_ext_data_in_ldir = MZ_TRUE; + break; + } + + pExtra_data += field_total_size; + extra_size_remaining -= field_total_size; + } while (extra_size_remaining); + + mz_zip_array_clear(pZip, &file_data_array); + } + + if (!pState->m_zip64) + { + /* Try to detect if the new archive will most likely wind up too big and bail early (+(sizeof(mz_uint32) * 4) is for the optional descriptor which could be present, +64 is a fudge factor). */ + /* We also check when the archive is finalized so this doesn't need to be perfect. */ + mz_uint64 approx_new_archive_size = cur_dst_file_ofs + num_alignment_padding_bytes + MZ_ZIP_LOCAL_DIR_HEADER_SIZE + src_archive_bytes_remaining + (sizeof(mz_uint32) * 4) + + pState->m_central_dir.m_size + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + src_central_dir_following_data_size + MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE + 64; + + if (approx_new_archive_size >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + } + + /* Write dest archive padding */ + if (!mz_zip_writer_write_zeros(pZip, cur_dst_file_ofs, num_alignment_padding_bytes)) + return MZ_FALSE; + + cur_dst_file_ofs += num_alignment_padding_bytes; + + local_dir_header_ofs = cur_dst_file_ofs; + if (pZip->m_file_offset_alignment) + { + MZ_ASSERT((local_dir_header_ofs & (pZip->m_file_offset_alignment - 1)) == 0); + } + + /* The original zip's local header+ext block doesn't change, even with zip64, so we can just copy it over to the dest zip */ + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_dst_file_ofs, pLocal_header, MZ_ZIP_LOCAL_DIR_HEADER_SIZE) != MZ_ZIP_LOCAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + cur_dst_file_ofs += MZ_ZIP_LOCAL_DIR_HEADER_SIZE; + + /* Copy over the source archive bytes to the dest archive, also ensure we have enough buf space to handle optional data descriptor */ + if (NULL == (pBuf = pZip->m_pAlloc(pZip->m_pAlloc_opaque, 1, (size_t)MZ_MAX(32U, MZ_MIN((mz_uint64)MZ_ZIP_MAX_IO_BUF_SIZE, src_archive_bytes_remaining))))) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + while (src_archive_bytes_remaining) + { + n = (mz_uint)MZ_MIN((mz_uint64)MZ_ZIP_MAX_IO_BUF_SIZE, src_archive_bytes_remaining); + if (pSource_zip->m_pRead(pSource_zip->m_pIO_opaque, cur_src_file_ofs, pBuf, n) != n) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + } + cur_src_file_ofs += n; + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_dst_file_ofs, pBuf, n) != n) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + cur_dst_file_ofs += n; + + src_archive_bytes_remaining -= n; + } + + /* Now deal with the optional data descriptor */ + bit_flags = MZ_READ_LE16(pLocal_header + MZ_ZIP_LDH_BIT_FLAG_OFS); + if (bit_flags & 8) + { + /* Copy data descriptor */ + if ((pSource_zip->m_pState->m_zip64) || (found_zip64_ext_data_in_ldir)) + { + /* src is zip64, dest must be zip64 */ + + /* name uint32_t's */ + /* id 1 (optional in zip64?) */ + /* crc 1 */ + /* comp_size 2 */ + /* uncomp_size 2 */ + if (pSource_zip->m_pRead(pSource_zip->m_pIO_opaque, cur_src_file_ofs, pBuf, (sizeof(mz_uint32) * 6)) != (sizeof(mz_uint32) * 6)) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + } + + n = sizeof(mz_uint32) * ((MZ_READ_LE32(pBuf) == MZ_ZIP_DATA_DESCRIPTOR_ID) ? 6 : 5); + } + else + { + /* src is NOT zip64 */ + mz_bool has_id; + + if (pSource_zip->m_pRead(pSource_zip->m_pIO_opaque, cur_src_file_ofs, pBuf, sizeof(mz_uint32) * 4) != sizeof(mz_uint32) * 4) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_READ_FAILED); + } + + has_id = (MZ_READ_LE32(pBuf) == MZ_ZIP_DATA_DESCRIPTOR_ID); + + if (pZip->m_pState->m_zip64) + { + /* dest is zip64, so upgrade the data descriptor */ + const mz_uint8 *pSrc_descriptor = (const mz_uint8 *)pBuf + (has_id ? sizeof(mz_uint32) : 0); + const mz_uint32 src_crc32 = MZ_READ_LE32(pSrc_descriptor); + const mz_uint64 src_comp_size = MZ_READ_LE32(pSrc_descriptor + sizeof(mz_uint32)); + const mz_uint64 src_uncomp_size = MZ_READ_LE32(pSrc_descriptor + 2 * sizeof(mz_uint32)); + + mz_write_le32((mz_uint8 *)pBuf, MZ_ZIP_DATA_DESCRIPTOR_ID); + mz_write_le32((mz_uint8 *)pBuf + sizeof(mz_uint32) * 1, src_crc32); + mz_write_le64((mz_uint8 *)pBuf + sizeof(mz_uint32) * 2, src_comp_size); + mz_write_le64((mz_uint8 *)pBuf + sizeof(mz_uint32) * 4, src_uncomp_size); + + n = sizeof(mz_uint32) * 6; + } + else + { + /* dest is NOT zip64, just copy it as-is */ + n = sizeof(mz_uint32) * (has_id ? 4 : 3); + } + } + + if (pZip->m_pWrite(pZip->m_pIO_opaque, cur_dst_file_ofs, pBuf, n) != n) + { + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + } + + cur_src_file_ofs += n; + cur_dst_file_ofs += n; + } + pZip->m_pFree(pZip->m_pAlloc_opaque, pBuf); + + /* Finally, add the new central dir header */ + orig_central_dir_size = pState->m_central_dir.m_size; + + memcpy(new_central_header, pSrc_central_header, MZ_ZIP_CENTRAL_DIR_HEADER_SIZE); + + if (pState->m_zip64) + { + /* This is the painful part: We need to write a new central dir header + ext block with updated zip64 fields, and ensure the old fields (if any) are not included. */ + const mz_uint8 *pSrc_ext = pSrc_central_header + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + src_filename_len; + mz_zip_array new_ext_block; + + mz_zip_array_init(&new_ext_block, sizeof(mz_uint8)); + + MZ_WRITE_LE32(new_central_header + MZ_ZIP_CDH_COMPRESSED_SIZE_OFS, MZ_UINT32_MAX); + MZ_WRITE_LE32(new_central_header + MZ_ZIP_CDH_DECOMPRESSED_SIZE_OFS, MZ_UINT32_MAX); + MZ_WRITE_LE32(new_central_header + MZ_ZIP_CDH_LOCAL_HEADER_OFS, MZ_UINT32_MAX); + + if (!mz_zip_writer_update_zip64_extension_block(&new_ext_block, pZip, pSrc_ext, src_ext_len, &src_file_stat.m_comp_size, &src_file_stat.m_uncomp_size, &local_dir_header_ofs, NULL)) + { + mz_zip_array_clear(pZip, &new_ext_block); + return MZ_FALSE; + } + + MZ_WRITE_LE16(new_central_header + MZ_ZIP_CDH_EXTRA_LEN_OFS, new_ext_block.m_size); + + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir, new_central_header, MZ_ZIP_CENTRAL_DIR_HEADER_SIZE)) + { + mz_zip_array_clear(pZip, &new_ext_block); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir, pSrc_central_header + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE, src_filename_len)) + { + mz_zip_array_clear(pZip, &new_ext_block); + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir, new_ext_block.m_p, new_ext_block.m_size)) + { + mz_zip_array_clear(pZip, &new_ext_block); + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir, pSrc_central_header + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE + src_filename_len + src_ext_len, src_comment_len)) + { + mz_zip_array_clear(pZip, &new_ext_block); + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + mz_zip_array_clear(pZip, &new_ext_block); + } + else + { + /* sanity checks */ + if (cur_dst_file_ofs > MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + + if (local_dir_header_ofs >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_ARCHIVE_TOO_LARGE); + + MZ_WRITE_LE32(new_central_header + MZ_ZIP_CDH_LOCAL_HEADER_OFS, local_dir_header_ofs); + + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir, new_central_header, MZ_ZIP_CENTRAL_DIR_HEADER_SIZE)) + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir, pSrc_central_header + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE, src_central_dir_following_data_size)) + { + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + } + + /* This shouldn't trigger unless we screwed up during the initial sanity checks */ + if (pState->m_central_dir.m_size >= MZ_UINT32_MAX) + { + /* TODO: Support central dirs >= 32-bits in size */ + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_UNSUPPORTED_CDIR_SIZE); + } + + n = (mz_uint32)orig_central_dir_size; + if (!mz_zip_array_push_back(pZip, &pState->m_central_dir_offsets, &n, 1)) + { + mz_zip_array_resize(pZip, &pState->m_central_dir, orig_central_dir_size, MZ_FALSE); + return mz_zip_set_error(pZip, MZ_ZIP_ALLOC_FAILED); + } + + pZip->m_total_files++; + pZip->m_archive_size = cur_dst_file_ofs; + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_finalize_archive(mz_zip_archive *pZip) + { + mz_zip_internal_state *pState; + mz_uint64 central_dir_ofs, central_dir_size; + mz_uint8 hdr[256]; + + if ((!pZip) || (!pZip->m_pState) || (pZip->m_zip_mode != MZ_ZIP_MODE_WRITING)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + pState = pZip->m_pState; + + if (pState->m_zip64) + { + if ((mz_uint64)pState->m_central_dir.m_size >= MZ_UINT32_MAX) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + else + { + if ((pZip->m_total_files > MZ_UINT16_MAX) || ((pZip->m_archive_size + pState->m_central_dir.m_size + MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) > MZ_UINT32_MAX)) + return mz_zip_set_error(pZip, MZ_ZIP_TOO_MANY_FILES); + } + + central_dir_ofs = 0; + central_dir_size = 0; + if (pZip->m_total_files) + { + /* Write central directory */ + central_dir_ofs = pZip->m_archive_size; + central_dir_size = pState->m_central_dir.m_size; + pZip->m_central_directory_file_ofs = central_dir_ofs; + if (pZip->m_pWrite(pZip->m_pIO_opaque, central_dir_ofs, pState->m_central_dir.m_p, (size_t)central_dir_size) != central_dir_size) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + pZip->m_archive_size += central_dir_size; + } + + if (pState->m_zip64) + { + /* Write zip64 end of central directory header */ + mz_uint64 rel_ofs_to_zip64_ecdr = pZip->m_archive_size; + + MZ_CLEAR_ARR(hdr); + MZ_WRITE_LE32(hdr + MZ_ZIP64_ECDH_SIG_OFS, MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIG); + MZ_WRITE_LE64(hdr + MZ_ZIP64_ECDH_SIZE_OF_RECORD_OFS, MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE - sizeof(mz_uint32) - sizeof(mz_uint64)); + MZ_WRITE_LE16(hdr + MZ_ZIP64_ECDH_VERSION_MADE_BY_OFS, 0x031E); /* TODO: always Unix */ + MZ_WRITE_LE16(hdr + MZ_ZIP64_ECDH_VERSION_NEEDED_OFS, 0x002D); + MZ_WRITE_LE64(hdr + MZ_ZIP64_ECDH_CDIR_NUM_ENTRIES_ON_DISK_OFS, pZip->m_total_files); + MZ_WRITE_LE64(hdr + MZ_ZIP64_ECDH_CDIR_TOTAL_ENTRIES_OFS, pZip->m_total_files); + MZ_WRITE_LE64(hdr + MZ_ZIP64_ECDH_CDIR_SIZE_OFS, central_dir_size); + MZ_WRITE_LE64(hdr + MZ_ZIP64_ECDH_CDIR_OFS_OFS, central_dir_ofs); + if (pZip->m_pWrite(pZip->m_pIO_opaque, pZip->m_archive_size, hdr, MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE) != MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + pZip->m_archive_size += MZ_ZIP64_END_OF_CENTRAL_DIR_HEADER_SIZE; + + /* Write zip64 end of central directory locator */ + MZ_CLEAR_ARR(hdr); + MZ_WRITE_LE32(hdr + MZ_ZIP64_ECDL_SIG_OFS, MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIG); + MZ_WRITE_LE64(hdr + MZ_ZIP64_ECDL_REL_OFS_TO_ZIP64_ECDR_OFS, rel_ofs_to_zip64_ecdr); + MZ_WRITE_LE32(hdr + MZ_ZIP64_ECDL_TOTAL_NUMBER_OF_DISKS_OFS, 1); + if (pZip->m_pWrite(pZip->m_pIO_opaque, pZip->m_archive_size, hdr, MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE) != MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + + pZip->m_archive_size += MZ_ZIP64_END_OF_CENTRAL_DIR_LOCATOR_SIZE; + } + + /* Write end of central directory record */ + MZ_CLEAR_ARR(hdr); + MZ_WRITE_LE32(hdr + MZ_ZIP_ECDH_SIG_OFS, MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIG); + MZ_WRITE_LE16(hdr + MZ_ZIP_ECDH_CDIR_NUM_ENTRIES_ON_DISK_OFS, MZ_MIN(MZ_UINT16_MAX, pZip->m_total_files)); + MZ_WRITE_LE16(hdr + MZ_ZIP_ECDH_CDIR_TOTAL_ENTRIES_OFS, MZ_MIN(MZ_UINT16_MAX, pZip->m_total_files)); + MZ_WRITE_LE32(hdr + MZ_ZIP_ECDH_CDIR_SIZE_OFS, MZ_MIN(MZ_UINT32_MAX, central_dir_size)); + MZ_WRITE_LE32(hdr + MZ_ZIP_ECDH_CDIR_OFS_OFS, MZ_MIN(MZ_UINT32_MAX, central_dir_ofs)); + + if (pZip->m_pWrite(pZip->m_pIO_opaque, pZip->m_archive_size, hdr, MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) != MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_WRITE_FAILED); + +#ifndef MINIZ_NO_STDIO + if ((pState->m_pFile) && (MZ_FFLUSH(pState->m_pFile) == EOF)) + return mz_zip_set_error(pZip, MZ_ZIP_FILE_CLOSE_FAILED); +#endif /* #ifndef MINIZ_NO_STDIO */ + + pZip->m_archive_size += MZ_ZIP_END_OF_CENTRAL_DIR_HEADER_SIZE; + + pZip->m_zip_mode = MZ_ZIP_MODE_WRITING_HAS_BEEN_FINALIZED; + return MZ_TRUE; + } + + mz_bool mz_zip_writer_finalize_heap_archive(mz_zip_archive *pZip, void **ppBuf, size_t *pSize) + { + if ((!ppBuf) || (!pSize)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + *ppBuf = NULL; + *pSize = 0; + + if ((!pZip) || (!pZip->m_pState)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (pZip->m_pWrite != mz_zip_heap_write_func) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + if (!mz_zip_writer_finalize_archive(pZip)) + return MZ_FALSE; + + *ppBuf = pZip->m_pState->m_pMem; + *pSize = pZip->m_pState->m_mem_size; + pZip->m_pState->m_pMem = NULL; + pZip->m_pState->m_mem_size = pZip->m_pState->m_mem_capacity = 0; + + return MZ_TRUE; + } + + mz_bool mz_zip_writer_end(mz_zip_archive *pZip) + { + return mz_zip_writer_end_internal(pZip, MZ_TRUE); + } + +#ifndef MINIZ_NO_STDIO + mz_bool mz_zip_add_mem_to_archive_file_in_place(const char *pZip_filename, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags) + { + return mz_zip_add_mem_to_archive_file_in_place_v2(pZip_filename, pArchive_name, pBuf, buf_size, pComment, comment_size, level_and_flags, NULL); + } + + mz_bool mz_zip_add_mem_to_archive_file_in_place_v2(const char *pZip_filename, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, mz_zip_error *pErr) + { + mz_bool status, created_new_archive = MZ_FALSE; + mz_zip_archive zip_archive; + struct MZ_FILE_STAT_STRUCT file_stat; + mz_zip_error actual_err = MZ_ZIP_NO_ERROR; + + mz_zip_zero_struct(&zip_archive); + if ((int)level_and_flags < 0) + level_and_flags = MZ_DEFAULT_LEVEL; + + if ((!pZip_filename) || (!pArchive_name) || ((buf_size) && (!pBuf)) || ((comment_size) && (!pComment)) || ((level_and_flags & 0xF) > MZ_UBER_COMPRESSION)) + { + if (pErr) + *pErr = MZ_ZIP_INVALID_PARAMETER; + return MZ_FALSE; + } + + if (!mz_zip_writer_validate_archive_name(pArchive_name)) + { + if (pErr) + *pErr = MZ_ZIP_INVALID_FILENAME; + return MZ_FALSE; + } + + /* Important: The regular non-64 bit version of stat() can fail here if the file is very large, which could cause the archive to be overwritten. */ + /* So be sure to compile with _LARGEFILE64_SOURCE 1 */ + if (MZ_FILE_STAT(pZip_filename, &file_stat) != 0) + { + /* Create a new archive. */ + if (!mz_zip_writer_init_file_v2(&zip_archive, pZip_filename, 0, level_and_flags)) + { + if (pErr) + *pErr = zip_archive.m_last_error; + return MZ_FALSE; + } + + created_new_archive = MZ_TRUE; + } + else + { + /* Append to an existing archive. */ + if (!mz_zip_reader_init_file_v2(&zip_archive, pZip_filename, level_and_flags | MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY | MZ_ZIP_FLAG_READ_ALLOW_WRITING, 0, 0)) + { + if (pErr) + *pErr = zip_archive.m_last_error; + return MZ_FALSE; + } + + if (!mz_zip_writer_init_from_reader_v2(&zip_archive, pZip_filename, level_and_flags | MZ_ZIP_FLAG_READ_ALLOW_WRITING)) + { + if (pErr) + *pErr = zip_archive.m_last_error; + + mz_zip_reader_end_internal(&zip_archive, MZ_FALSE); + + return MZ_FALSE; + } + } + + status = mz_zip_writer_add_mem_ex(&zip_archive, pArchive_name, pBuf, buf_size, pComment, comment_size, level_and_flags, 0, 0); + actual_err = zip_archive.m_last_error; + + /* Always finalize, even if adding failed for some reason, so we have a valid central directory. (This may not always succeed, but we can try.) */ + if (!mz_zip_writer_finalize_archive(&zip_archive)) + { + if (!actual_err) + actual_err = zip_archive.m_last_error; + + status = MZ_FALSE; + } + + if (!mz_zip_writer_end_internal(&zip_archive, status)) + { + if (!actual_err) + actual_err = zip_archive.m_last_error; + + status = MZ_FALSE; + } + + if ((!status) && (created_new_archive)) + { + /* It's a new archive and something went wrong, so just delete it. */ + int ignoredStatus = MZ_DELETE_FILE(pZip_filename); + (void)ignoredStatus; + } + + if (pErr) + *pErr = actual_err; + + return status; + } + + void *mz_zip_extract_archive_file_to_heap_v2(const char *pZip_filename, const char *pArchive_name, const char *pComment, size_t *pSize, mz_uint flags, mz_zip_error *pErr) + { + mz_uint32 file_index; + mz_zip_archive zip_archive; + void *p = NULL; + + if (pSize) + *pSize = 0; + + if ((!pZip_filename) || (!pArchive_name)) + { + if (pErr) + *pErr = MZ_ZIP_INVALID_PARAMETER; + + return NULL; + } + + mz_zip_zero_struct(&zip_archive); + if (!mz_zip_reader_init_file_v2(&zip_archive, pZip_filename, flags | MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY, 0, 0)) + { + if (pErr) + *pErr = zip_archive.m_last_error; + + return NULL; + } + + if (mz_zip_reader_locate_file_v2(&zip_archive, pArchive_name, pComment, flags, &file_index)) + { + p = mz_zip_reader_extract_to_heap(&zip_archive, file_index, pSize, flags); + } + + mz_zip_reader_end_internal(&zip_archive, p != NULL); + + if (pErr) + *pErr = zip_archive.m_last_error; + + return p; + } + + void *mz_zip_extract_archive_file_to_heap(const char *pZip_filename, const char *pArchive_name, size_t *pSize, mz_uint flags) + { + return mz_zip_extract_archive_file_to_heap_v2(pZip_filename, pArchive_name, NULL, pSize, flags, NULL); + } + +#endif /* #ifndef MINIZ_NO_STDIO */ + +#endif /* #ifndef MINIZ_NO_ARCHIVE_WRITING_APIS */ + + /* ------------------- Misc utils */ + + mz_zip_mode mz_zip_get_mode(mz_zip_archive *pZip) + { + return pZip ? pZip->m_zip_mode : MZ_ZIP_MODE_INVALID; + } + + mz_zip_type mz_zip_get_type(mz_zip_archive *pZip) + { + return pZip ? pZip->m_zip_type : MZ_ZIP_TYPE_INVALID; + } + + mz_zip_error mz_zip_set_last_error(mz_zip_archive *pZip, mz_zip_error err_num) + { + mz_zip_error prev_err; + + if (!pZip) + return MZ_ZIP_INVALID_PARAMETER; + + prev_err = pZip->m_last_error; + + pZip->m_last_error = err_num; + return prev_err; + } + + mz_zip_error mz_zip_peek_last_error(mz_zip_archive *pZip) + { + if (!pZip) + return MZ_ZIP_INVALID_PARAMETER; + + return pZip->m_last_error; + } + + mz_zip_error mz_zip_clear_last_error(mz_zip_archive *pZip) + { + return mz_zip_set_last_error(pZip, MZ_ZIP_NO_ERROR); + } + + mz_zip_error mz_zip_get_last_error(mz_zip_archive *pZip) + { + mz_zip_error prev_err; + + if (!pZip) + return MZ_ZIP_INVALID_PARAMETER; + + prev_err = pZip->m_last_error; + + pZip->m_last_error = MZ_ZIP_NO_ERROR; + return prev_err; + } + + const char *mz_zip_get_error_string(mz_zip_error mz_err) + { + switch (mz_err) + { + case MZ_ZIP_NO_ERROR: + return "no error"; + case MZ_ZIP_UNDEFINED_ERROR: + return "undefined error"; + case MZ_ZIP_TOO_MANY_FILES: + return "too many files"; + case MZ_ZIP_FILE_TOO_LARGE: + return "file too large"; + case MZ_ZIP_UNSUPPORTED_METHOD: + return "unsupported method"; + case MZ_ZIP_UNSUPPORTED_ENCRYPTION: + return "unsupported encryption"; + case MZ_ZIP_UNSUPPORTED_FEATURE: + return "unsupported feature"; + case MZ_ZIP_FAILED_FINDING_CENTRAL_DIR: + return "failed finding central directory"; + case MZ_ZIP_NOT_AN_ARCHIVE: + return "not a ZIP archive"; + case MZ_ZIP_INVALID_HEADER_OR_CORRUPTED: + return "invalid header or archive is corrupted"; + case MZ_ZIP_UNSUPPORTED_MULTIDISK: + return "unsupported multidisk archive"; + case MZ_ZIP_DECOMPRESSION_FAILED: + return "decompression failed or archive is corrupted"; + case MZ_ZIP_COMPRESSION_FAILED: + return "compression failed"; + case MZ_ZIP_UNEXPECTED_DECOMPRESSED_SIZE: + return "unexpected decompressed size"; + case MZ_ZIP_CRC_CHECK_FAILED: + return "CRC-32 check failed"; + case MZ_ZIP_UNSUPPORTED_CDIR_SIZE: + return "unsupported central directory size"; + case MZ_ZIP_ALLOC_FAILED: + return "allocation failed"; + case MZ_ZIP_FILE_OPEN_FAILED: + return "file open failed"; + case MZ_ZIP_FILE_CREATE_FAILED: + return "file create failed"; + case MZ_ZIP_FILE_WRITE_FAILED: + return "file write failed"; + case MZ_ZIP_FILE_READ_FAILED: + return "file read failed"; + case MZ_ZIP_FILE_CLOSE_FAILED: + return "file close failed"; + case MZ_ZIP_FILE_SEEK_FAILED: + return "file seek failed"; + case MZ_ZIP_FILE_STAT_FAILED: + return "file stat failed"; + case MZ_ZIP_INVALID_PARAMETER: + return "invalid parameter"; + case MZ_ZIP_INVALID_FILENAME: + return "invalid filename"; + case MZ_ZIP_BUF_TOO_SMALL: + return "buffer too small"; + case MZ_ZIP_INTERNAL_ERROR: + return "internal error"; + case MZ_ZIP_FILE_NOT_FOUND: + return "file not found"; + case MZ_ZIP_ARCHIVE_TOO_LARGE: + return "archive is too large"; + case MZ_ZIP_VALIDATION_FAILED: + return "validation failed"; + case MZ_ZIP_WRITE_CALLBACK_FAILED: + return "write callback failed"; + case MZ_ZIP_TOTAL_ERRORS: + return "total errors"; + default: + break; + } + + return "unknown error"; + } + + /* Note: Just because the archive is not zip64 doesn't necessarily mean it doesn't have Zip64 extended information extra field, argh. */ + mz_bool mz_zip_is_zip64(mz_zip_archive *pZip) + { + if ((!pZip) || (!pZip->m_pState)) + return MZ_FALSE; + + return pZip->m_pState->m_zip64; + } + + size_t mz_zip_get_central_dir_size(mz_zip_archive *pZip) + { + if ((!pZip) || (!pZip->m_pState)) + return 0; + + return pZip->m_pState->m_central_dir.m_size; + } + + mz_uint mz_zip_reader_get_num_files(mz_zip_archive *pZip) + { + return pZip ? pZip->m_total_files : 0; + } + + mz_uint64 mz_zip_get_archive_size(mz_zip_archive *pZip) + { + if (!pZip) + return 0; + return pZip->m_archive_size; + } + + mz_uint64 mz_zip_get_archive_file_start_offset(mz_zip_archive *pZip) + { + if ((!pZip) || (!pZip->m_pState)) + return 0; + return pZip->m_pState->m_file_archive_start_ofs; + } + + MZ_FILE *mz_zip_get_cfile(mz_zip_archive *pZip) + { + if ((!pZip) || (!pZip->m_pState)) + return 0; + return pZip->m_pState->m_pFile; + } + + size_t mz_zip_read_archive_data(mz_zip_archive *pZip, mz_uint64 file_ofs, void *pBuf, size_t n) + { + if ((!pZip) || (!pZip->m_pState) || (!pBuf) || (!pZip->m_pRead)) + return mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + + return pZip->m_pRead(pZip->m_pIO_opaque, file_ofs, pBuf, n); + } + + mz_uint mz_zip_reader_get_filename(mz_zip_archive *pZip, mz_uint file_index, char *pFilename, mz_uint filename_buf_size) + { + mz_uint n; + const mz_uint8 *p = mz_zip_get_cdh(pZip, file_index); + if (!p) + { + if (filename_buf_size) + pFilename[0] = '\0'; + mz_zip_set_error(pZip, MZ_ZIP_INVALID_PARAMETER); + return 0; + } + n = MZ_READ_LE16(p + MZ_ZIP_CDH_FILENAME_LEN_OFS); + if (filename_buf_size) + { + n = MZ_MIN(n, filename_buf_size - 1); + memcpy(pFilename, p + MZ_ZIP_CENTRAL_DIR_HEADER_SIZE, n); + pFilename[n] = '\0'; + } + return n + 1; + } + + mz_bool mz_zip_reader_file_stat(mz_zip_archive *pZip, mz_uint file_index, mz_zip_archive_file_stat *pStat) + { + return mz_zip_file_stat_internal(pZip, file_index, mz_zip_get_cdh(pZip, file_index), pStat, NULL); + } + + mz_bool mz_zip_end(mz_zip_archive *pZip) + { + if (!pZip) + return MZ_FALSE; + + if (pZip->m_zip_mode == MZ_ZIP_MODE_READING) + return mz_zip_reader_end(pZip); +#ifndef MINIZ_NO_ARCHIVE_WRITING_APIS + else if ((pZip->m_zip_mode == MZ_ZIP_MODE_WRITING) || (pZip->m_zip_mode == MZ_ZIP_MODE_WRITING_HAS_BEEN_FINALIZED)) + return mz_zip_writer_end(pZip); +#endif + + return MZ_FALSE; + } + +#ifdef __cplusplus +} +#endif + +#endif /*#ifndef MINIZ_NO_ARCHIVE_APIS*/ diff --git a/src/helpers/miniz/miniz_zip.h b/src/helpers/miniz/miniz_zip.h new file mode 100644 index 000000000..ccdb99042 --- /dev/null +++ b/src/helpers/miniz/miniz_zip.h @@ -0,0 +1,454 @@ + +#pragma once +#include "miniz_common.h" + +/* ------------------- ZIP archive reading/writing */ + +#ifndef MINIZ_NO_ARCHIVE_APIS + +#ifdef __cplusplus +extern "C" +{ +#endif + + enum + { + /* Note: These enums can be reduced as needed to save memory or stack space - they are pretty conservative. */ + MZ_ZIP_MAX_IO_BUF_SIZE = 64 * 1024, + MZ_ZIP_MAX_ARCHIVE_FILENAME_SIZE = 512, + MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE = 512 + }; + + typedef struct + { + /* Central directory file index. */ + mz_uint32 m_file_index; + + /* Byte offset of this entry in the archive's central directory. Note we currently only support up to UINT_MAX or less bytes in the central dir. */ + mz_uint64 m_central_dir_ofs; + + /* These fields are copied directly from the zip's central dir. */ + mz_uint16 m_version_made_by; + mz_uint16 m_version_needed; + mz_uint16 m_bit_flag; + mz_uint16 m_method; + + /* CRC-32 of uncompressed data. */ + mz_uint32 m_crc32; + + /* File's compressed size. */ + mz_uint64 m_comp_size; + + /* File's uncompressed size. Note, I've seen some old archives where directory entries had 512 bytes for their uncompressed sizes, but when you try to unpack them you actually get 0 bytes. */ + mz_uint64 m_uncomp_size; + + /* Zip internal and external file attributes. */ + mz_uint16 m_internal_attr; + mz_uint32 m_external_attr; + + /* Entry's local header file offset in bytes. */ + mz_uint64 m_local_header_ofs; + + /* Size of comment in bytes. */ + mz_uint32 m_comment_size; + + /* MZ_TRUE if the entry appears to be a directory. */ + mz_bool m_is_directory; + + /* MZ_TRUE if the entry uses encryption/strong encryption (which miniz_zip doesn't support) */ + mz_bool m_is_encrypted; + + /* MZ_TRUE if the file is not encrypted, a patch file, and if it uses a compression method we support. */ + mz_bool m_is_supported; + + /* Filename. If string ends in '/' it's a subdirectory entry. */ + /* Guaranteed to be zero terminated, may be truncated to fit. */ + char m_filename[MZ_ZIP_MAX_ARCHIVE_FILENAME_SIZE]; + + /* Comment field. */ + /* Guaranteed to be zero terminated, may be truncated to fit. */ + char m_comment[MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE]; + +#ifdef MINIZ_NO_TIME + MZ_TIME_T m_padding; +#else + MZ_TIME_T m_time; +#endif + } mz_zip_archive_file_stat; + + typedef size_t (*mz_file_read_func)(void *pOpaque, mz_uint64 file_ofs, void *pBuf, size_t n); + typedef size_t (*mz_file_write_func)(void *pOpaque, mz_uint64 file_ofs, const void *pBuf, size_t n); + typedef mz_bool (*mz_file_needs_keepalive)(void *pOpaque); + + struct mz_zip_internal_state_tag; + typedef struct mz_zip_internal_state_tag mz_zip_internal_state; + + typedef enum + { + MZ_ZIP_MODE_INVALID = 0, + MZ_ZIP_MODE_READING = 1, + MZ_ZIP_MODE_WRITING = 2, + MZ_ZIP_MODE_WRITING_HAS_BEEN_FINALIZED = 3 + } mz_zip_mode; + + typedef enum + { + MZ_ZIP_FLAG_CASE_SENSITIVE = 0x0100, + MZ_ZIP_FLAG_IGNORE_PATH = 0x0200, + MZ_ZIP_FLAG_COMPRESSED_DATA = 0x0400, + MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY = 0x0800, + MZ_ZIP_FLAG_VALIDATE_LOCATE_FILE_FLAG = 0x1000, /* if enabled, mz_zip_reader_locate_file() will be called on each file as its validated to ensure the func finds the file in the central dir (intended for testing) */ + MZ_ZIP_FLAG_VALIDATE_HEADERS_ONLY = 0x2000, /* validate the local headers, but don't decompress the entire file and check the crc32 */ + MZ_ZIP_FLAG_WRITE_ZIP64 = 0x4000, /* always use the zip64 file format, instead of the original zip file format with automatic switch to zip64. Use as flags parameter with mz_zip_writer_init*_v2 */ + MZ_ZIP_FLAG_WRITE_ALLOW_READING = 0x8000, + MZ_ZIP_FLAG_ASCII_FILENAME = 0x10000, + /*After adding a compressed file, seek back + to local file header and set the correct sizes*/ + MZ_ZIP_FLAG_WRITE_HEADER_SET_SIZE = 0x20000, + MZ_ZIP_FLAG_READ_ALLOW_WRITING = 0x40000 + } mz_zip_flags; + + typedef enum + { + MZ_ZIP_TYPE_INVALID = 0, + MZ_ZIP_TYPE_USER, + MZ_ZIP_TYPE_MEMORY, + MZ_ZIP_TYPE_HEAP, + MZ_ZIP_TYPE_FILE, + MZ_ZIP_TYPE_CFILE, + MZ_ZIP_TOTAL_TYPES + } mz_zip_type; + + /* miniz error codes. Be sure to update mz_zip_get_error_string() if you add or modify this enum. */ + typedef enum + { + MZ_ZIP_NO_ERROR = 0, + MZ_ZIP_UNDEFINED_ERROR, + MZ_ZIP_TOO_MANY_FILES, + MZ_ZIP_FILE_TOO_LARGE, + MZ_ZIP_UNSUPPORTED_METHOD, + MZ_ZIP_UNSUPPORTED_ENCRYPTION, + MZ_ZIP_UNSUPPORTED_FEATURE, + MZ_ZIP_FAILED_FINDING_CENTRAL_DIR, + MZ_ZIP_NOT_AN_ARCHIVE, + MZ_ZIP_INVALID_HEADER_OR_CORRUPTED, + MZ_ZIP_UNSUPPORTED_MULTIDISK, + MZ_ZIP_DECOMPRESSION_FAILED, + MZ_ZIP_COMPRESSION_FAILED, + MZ_ZIP_UNEXPECTED_DECOMPRESSED_SIZE, + MZ_ZIP_CRC_CHECK_FAILED, + MZ_ZIP_UNSUPPORTED_CDIR_SIZE, + MZ_ZIP_ALLOC_FAILED, + MZ_ZIP_FILE_OPEN_FAILED, + MZ_ZIP_FILE_CREATE_FAILED, + MZ_ZIP_FILE_WRITE_FAILED, + MZ_ZIP_FILE_READ_FAILED, + MZ_ZIP_FILE_CLOSE_FAILED, + MZ_ZIP_FILE_SEEK_FAILED, + MZ_ZIP_FILE_STAT_FAILED, + MZ_ZIP_INVALID_PARAMETER, + MZ_ZIP_INVALID_FILENAME, + MZ_ZIP_BUF_TOO_SMALL, + MZ_ZIP_INTERNAL_ERROR, + MZ_ZIP_FILE_NOT_FOUND, + MZ_ZIP_ARCHIVE_TOO_LARGE, + MZ_ZIP_VALIDATION_FAILED, + MZ_ZIP_WRITE_CALLBACK_FAILED, + MZ_ZIP_TOTAL_ERRORS + } mz_zip_error; + + typedef struct + { + mz_uint64 m_archive_size; + mz_uint64 m_central_directory_file_ofs; + + /* We only support up to UINT32_MAX files in zip64 mode. */ + mz_uint32 m_total_files; + mz_zip_mode m_zip_mode; + mz_zip_type m_zip_type; + mz_zip_error m_last_error; + + mz_uint64 m_file_offset_alignment; + + mz_alloc_func m_pAlloc; + mz_free_func m_pFree; + mz_realloc_func m_pRealloc; + void *m_pAlloc_opaque; + + mz_file_read_func m_pRead; + mz_file_write_func m_pWrite; + mz_file_needs_keepalive m_pNeeds_keepalive; + void *m_pIO_opaque; + + mz_zip_internal_state *m_pState; + + } mz_zip_archive; + + typedef struct + { + mz_zip_archive *pZip; + mz_uint flags; + + int status; + + mz_uint64 read_buf_size, read_buf_ofs, read_buf_avail, comp_remaining, out_buf_ofs, cur_file_ofs; + mz_zip_archive_file_stat file_stat; + void *pRead_buf; + void *pWrite_buf; + + size_t out_blk_remain; + + tinfl_decompressor inflator; + +#ifdef MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS + mz_uint padding; +#else + mz_uint file_crc32; +#endif + + } mz_zip_reader_extract_iter_state; + + /* -------- ZIP reading */ + + /* Inits a ZIP archive reader. */ + /* These functions read and validate the archive's central directory. */ + MINIZ_EXPORT mz_bool mz_zip_reader_init(mz_zip_archive *pZip, mz_uint64 size, mz_uint flags); + + MINIZ_EXPORT mz_bool mz_zip_reader_init_mem(mz_zip_archive *pZip, const void *pMem, size_t size, mz_uint flags); + +#ifndef MINIZ_NO_STDIO + /* Read a archive from a disk file. */ + /* file_start_ofs is the file offset where the archive actually begins, or 0. */ + /* actual_archive_size is the true total size of the archive, which may be smaller than the file's actual size on disk. If zero the entire file is treated as the archive. */ + MINIZ_EXPORT mz_bool mz_zip_reader_init_file(mz_zip_archive *pZip, const char *pFilename, mz_uint32 flags); + MINIZ_EXPORT mz_bool mz_zip_reader_init_file_v2(mz_zip_archive *pZip, const char *pFilename, mz_uint flags, mz_uint64 file_start_ofs, mz_uint64 archive_size); + + /* Read an archive from an already opened FILE, beginning at the current file position. */ + /* The archive is assumed to be archive_size bytes long. If archive_size is 0, then the entire rest of the file is assumed to contain the archive. */ + /* The FILE will NOT be closed when mz_zip_reader_end() is called. */ + MINIZ_EXPORT mz_bool mz_zip_reader_init_cfile(mz_zip_archive *pZip, MZ_FILE *pFile, mz_uint64 archive_size, mz_uint flags); +#endif + + /* Ends archive reading, freeing all allocations, and closing the input archive file if mz_zip_reader_init_file() was used. */ + MINIZ_EXPORT mz_bool mz_zip_reader_end(mz_zip_archive *pZip); + + /* -------- ZIP reading or writing */ + + /* Clears a mz_zip_archive struct to all zeros. */ + /* Important: This must be done before passing the struct to any mz_zip functions. */ + MINIZ_EXPORT void mz_zip_zero_struct(mz_zip_archive *pZip); + + MINIZ_EXPORT mz_zip_mode mz_zip_get_mode(mz_zip_archive *pZip); + MINIZ_EXPORT mz_zip_type mz_zip_get_type(mz_zip_archive *pZip); + + /* Returns the total number of files in the archive. */ + MINIZ_EXPORT mz_uint mz_zip_reader_get_num_files(mz_zip_archive *pZip); + + MINIZ_EXPORT mz_uint64 mz_zip_get_archive_size(mz_zip_archive *pZip); + MINIZ_EXPORT mz_uint64 mz_zip_get_archive_file_start_offset(mz_zip_archive *pZip); + MINIZ_EXPORT MZ_FILE *mz_zip_get_cfile(mz_zip_archive *pZip); + + /* Reads n bytes of raw archive data, starting at file offset file_ofs, to pBuf. */ + MINIZ_EXPORT size_t mz_zip_read_archive_data(mz_zip_archive *pZip, mz_uint64 file_ofs, void *pBuf, size_t n); + + /* All mz_zip funcs set the m_last_error field in the mz_zip_archive struct. These functions retrieve/manipulate this field. */ + /* Note that the m_last_error functionality is not thread safe. */ + MINIZ_EXPORT mz_zip_error mz_zip_set_last_error(mz_zip_archive *pZip, mz_zip_error err_num); + MINIZ_EXPORT mz_zip_error mz_zip_peek_last_error(mz_zip_archive *pZip); + MINIZ_EXPORT mz_zip_error mz_zip_clear_last_error(mz_zip_archive *pZip); + MINIZ_EXPORT mz_zip_error mz_zip_get_last_error(mz_zip_archive *pZip); + MINIZ_EXPORT const char *mz_zip_get_error_string(mz_zip_error mz_err); + + /* MZ_TRUE if the archive file entry is a directory entry. */ + MINIZ_EXPORT mz_bool mz_zip_reader_is_file_a_directory(mz_zip_archive *pZip, mz_uint file_index); + + /* MZ_TRUE if the file is encrypted/strong encrypted. */ + MINIZ_EXPORT mz_bool mz_zip_reader_is_file_encrypted(mz_zip_archive *pZip, mz_uint file_index); + + /* MZ_TRUE if the compression method is supported, and the file is not encrypted, and the file is not a compressed patch file. */ + MINIZ_EXPORT mz_bool mz_zip_reader_is_file_supported(mz_zip_archive *pZip, mz_uint file_index); + + /* Retrieves the filename of an archive file entry. */ + /* Returns the number of bytes written to pFilename, or if filename_buf_size is 0 this function returns the number of bytes needed to fully store the filename. */ + MINIZ_EXPORT mz_uint mz_zip_reader_get_filename(mz_zip_archive *pZip, mz_uint file_index, char *pFilename, mz_uint filename_buf_size); + + /* Attempts to locates a file in the archive's central directory. */ + /* Valid flags: MZ_ZIP_FLAG_CASE_SENSITIVE, MZ_ZIP_FLAG_IGNORE_PATH */ + /* Returns -1 if the file cannot be found. */ + MINIZ_EXPORT int mz_zip_reader_locate_file(mz_zip_archive *pZip, const char *pName, const char *pComment, mz_uint flags); + MINIZ_EXPORT mz_bool mz_zip_reader_locate_file_v2(mz_zip_archive *pZip, const char *pName, const char *pComment, mz_uint flags, mz_uint32 *file_index); + + /* Returns detailed information about an archive file entry. */ + MINIZ_EXPORT mz_bool mz_zip_reader_file_stat(mz_zip_archive *pZip, mz_uint file_index, mz_zip_archive_file_stat *pStat); + + /* MZ_TRUE if the file is in zip64 format. */ + /* A file is considered zip64 if it contained a zip64 end of central directory marker, or if it contained any zip64 extended file information fields in the central directory. */ + MINIZ_EXPORT mz_bool mz_zip_is_zip64(mz_zip_archive *pZip); + + /* Returns the total central directory size in bytes. */ + /* The current max supported size is <= MZ_UINT32_MAX. */ + MINIZ_EXPORT size_t mz_zip_get_central_dir_size(mz_zip_archive *pZip); + + /* Extracts a archive file to a memory buffer using no memory allocation. */ + /* There must be at least enough room on the stack to store the inflator's state (~34KB or so). */ + MINIZ_EXPORT mz_bool mz_zip_reader_extract_to_mem_no_alloc(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size); + MINIZ_EXPORT mz_bool mz_zip_reader_extract_file_to_mem_no_alloc(mz_zip_archive *pZip, const char *pFilename, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size); + + /* Extracts a archive file to a memory buffer. */ + MINIZ_EXPORT mz_bool mz_zip_reader_extract_to_mem(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags); + MINIZ_EXPORT mz_bool mz_zip_reader_extract_file_to_mem(mz_zip_archive *pZip, const char *pFilename, void *pBuf, size_t buf_size, mz_uint flags); + + /* Extracts a archive file to a dynamically allocated heap buffer. */ + /* The memory will be allocated via the mz_zip_archive's alloc/realloc functions. */ + /* Returns NULL and sets the last error on failure. */ + MINIZ_EXPORT void *mz_zip_reader_extract_to_heap(mz_zip_archive *pZip, mz_uint file_index, size_t *pSize, mz_uint flags); + MINIZ_EXPORT void *mz_zip_reader_extract_file_to_heap(mz_zip_archive *pZip, const char *pFilename, size_t *pSize, mz_uint flags); + + /* Extracts a archive file using a callback function to output the file's data. */ + MINIZ_EXPORT mz_bool mz_zip_reader_extract_to_callback(mz_zip_archive *pZip, mz_uint file_index, mz_file_write_func pCallback, void *pOpaque, mz_uint flags); + MINIZ_EXPORT mz_bool mz_zip_reader_extract_file_to_callback(mz_zip_archive *pZip, const char *pFilename, mz_file_write_func pCallback, void *pOpaque, mz_uint flags); + + /* Extract a file iteratively */ + MINIZ_EXPORT mz_zip_reader_extract_iter_state *mz_zip_reader_extract_iter_new(mz_zip_archive *pZip, mz_uint file_index, mz_uint flags); + MINIZ_EXPORT mz_zip_reader_extract_iter_state *mz_zip_reader_extract_file_iter_new(mz_zip_archive *pZip, const char *pFilename, mz_uint flags); + MINIZ_EXPORT size_t mz_zip_reader_extract_iter_read(mz_zip_reader_extract_iter_state *pState, void *pvBuf, size_t buf_size); + MINIZ_EXPORT mz_bool mz_zip_reader_extract_iter_free(mz_zip_reader_extract_iter_state *pState); + +#ifndef MINIZ_NO_STDIO + /* Extracts a archive file to a disk file and sets its last accessed and modified times. */ + /* This function only extracts files, not archive directory records. */ + MINIZ_EXPORT mz_bool mz_zip_reader_extract_to_file(mz_zip_archive *pZip, mz_uint file_index, const char *pDst_filename, mz_uint flags); + MINIZ_EXPORT mz_bool mz_zip_reader_extract_file_to_file(mz_zip_archive *pZip, const char *pArchive_filename, const char *pDst_filename, mz_uint flags); + + /* Extracts a archive file starting at the current position in the destination FILE stream. */ + MINIZ_EXPORT mz_bool mz_zip_reader_extract_to_cfile(mz_zip_archive *pZip, mz_uint file_index, MZ_FILE *File, mz_uint flags); + MINIZ_EXPORT mz_bool mz_zip_reader_extract_file_to_cfile(mz_zip_archive *pZip, const char *pArchive_filename, MZ_FILE *pFile, mz_uint flags); +#endif + +#if 0 +/* TODO */ + typedef void *mz_zip_streaming_extract_state_ptr; + mz_zip_streaming_extract_state_ptr mz_zip_streaming_extract_begin(mz_zip_archive *pZip, mz_uint file_index, mz_uint flags); + mz_uint64 mz_zip_streaming_extract_get_size(mz_zip_archive *pZip, mz_zip_streaming_extract_state_ptr pState); + mz_uint64 mz_zip_streaming_extract_get_cur_ofs(mz_zip_archive *pZip, mz_zip_streaming_extract_state_ptr pState); + mz_bool mz_zip_streaming_extract_seek(mz_zip_archive *pZip, mz_zip_streaming_extract_state_ptr pState, mz_uint64 new_ofs); + size_t mz_zip_streaming_extract_read(mz_zip_archive *pZip, mz_zip_streaming_extract_state_ptr pState, void *pBuf, size_t buf_size); + mz_bool mz_zip_streaming_extract_end(mz_zip_archive *pZip, mz_zip_streaming_extract_state_ptr pState); +#endif + + /* This function compares the archive's local headers, the optional local zip64 extended information block, and the optional descriptor following the compressed data vs. the data in the central directory. */ + /* It also validates that each file can be successfully uncompressed unless the MZ_ZIP_FLAG_VALIDATE_HEADERS_ONLY is specified. */ + MINIZ_EXPORT mz_bool mz_zip_validate_file(mz_zip_archive *pZip, mz_uint file_index, mz_uint flags); + + /* Validates an entire archive by calling mz_zip_validate_file() on each file. */ + MINIZ_EXPORT mz_bool mz_zip_validate_archive(mz_zip_archive *pZip, mz_uint flags); + + /* Misc utils/helpers, valid for ZIP reading or writing */ + MINIZ_EXPORT mz_bool mz_zip_validate_mem_archive(const void *pMem, size_t size, mz_uint flags, mz_zip_error *pErr); +#ifndef MINIZ_NO_STDIO + MINIZ_EXPORT mz_bool mz_zip_validate_file_archive(const char *pFilename, mz_uint flags, mz_zip_error *pErr); +#endif + + /* Universal end function - calls either mz_zip_reader_end() or mz_zip_writer_end(). */ + MINIZ_EXPORT mz_bool mz_zip_end(mz_zip_archive *pZip); + + /* -------- ZIP writing */ + +#ifndef MINIZ_NO_ARCHIVE_WRITING_APIS + + /* Inits a ZIP archive writer. */ + /*Set pZip->m_pWrite (and pZip->m_pIO_opaque) before calling mz_zip_writer_init or mz_zip_writer_init_v2*/ + /*The output is streamable, i.e. file_ofs in mz_file_write_func always increases only by n*/ + MINIZ_EXPORT mz_bool mz_zip_writer_init(mz_zip_archive *pZip, mz_uint64 existing_size); + MINIZ_EXPORT mz_bool mz_zip_writer_init_v2(mz_zip_archive *pZip, mz_uint64 existing_size, mz_uint flags); + + MINIZ_EXPORT mz_bool mz_zip_writer_init_heap(mz_zip_archive *pZip, size_t size_to_reserve_at_beginning, size_t initial_allocation_size); + MINIZ_EXPORT mz_bool mz_zip_writer_init_heap_v2(mz_zip_archive *pZip, size_t size_to_reserve_at_beginning, size_t initial_allocation_size, mz_uint flags); + +#ifndef MINIZ_NO_STDIO + MINIZ_EXPORT mz_bool mz_zip_writer_init_file(mz_zip_archive *pZip, const char *pFilename, mz_uint64 size_to_reserve_at_beginning); + MINIZ_EXPORT mz_bool mz_zip_writer_init_file_v2(mz_zip_archive *pZip, const char *pFilename, mz_uint64 size_to_reserve_at_beginning, mz_uint flags); + MINIZ_EXPORT mz_bool mz_zip_writer_init_cfile(mz_zip_archive *pZip, MZ_FILE *pFile, mz_uint flags); +#endif + + /* Converts a ZIP archive reader object into a writer object, to allow efficient in-place file appends to occur on an existing archive. */ + /* For archives opened using mz_zip_reader_init_file, pFilename must be the archive's filename so it can be reopened for writing. If the file can't be reopened, mz_zip_reader_end() will be called. */ + /* For archives opened using mz_zip_reader_init_mem, the memory block must be growable using the realloc callback (which defaults to realloc unless you've overridden it). */ + /* Finally, for archives opened using mz_zip_reader_init, the mz_zip_archive's user provided m_pWrite function cannot be NULL. */ + /* Note: In-place archive modification is not recommended unless you know what you're doing, because if execution stops or something goes wrong before */ + /* the archive is finalized the file's central directory will be hosed. */ + MINIZ_EXPORT mz_bool mz_zip_writer_init_from_reader(mz_zip_archive *pZip, const char *pFilename); + MINIZ_EXPORT mz_bool mz_zip_writer_init_from_reader_v2(mz_zip_archive *pZip, const char *pFilename, mz_uint flags); + + /* Adds the contents of a memory buffer to an archive. These functions record the current local time into the archive. */ + /* To add a directory entry, call this method with an archive name ending in a forwardslash with an empty buffer. */ + /* level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION. */ + MINIZ_EXPORT mz_bool mz_zip_writer_add_mem(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, mz_uint level_and_flags); + + /* Like mz_zip_writer_add_mem(), except you can specify a file comment field, and optionally supply the function with already compressed data. */ + /* uncomp_size/uncomp_crc32 are only used if the MZ_ZIP_FLAG_COMPRESSED_DATA flag is specified. */ + MINIZ_EXPORT mz_bool mz_zip_writer_add_mem_ex(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, + mz_uint64 uncomp_size, mz_uint32 uncomp_crc32); + + MINIZ_EXPORT mz_bool mz_zip_writer_add_mem_ex_v2(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, + mz_uint64 uncomp_size, mz_uint32 uncomp_crc32, MZ_TIME_T *last_modified, const char *user_extra_data_local, mz_uint user_extra_data_local_len, + const char *user_extra_data_central, mz_uint user_extra_data_central_len); + + /* Adds the contents of a file to an archive. This function also records the disk file's modified time into the archive. */ + /* File data is supplied via a read callback function. User mz_zip_writer_add_(c)file to add a file directly.*/ + MINIZ_EXPORT mz_bool mz_zip_writer_add_read_buf_callback(mz_zip_archive *pZip, const char *pArchive_name, mz_file_read_func read_callback, void *callback_opaque, mz_uint64 max_size, + const MZ_TIME_T *pFile_time, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, const char *user_extra_data_local, mz_uint user_extra_data_local_len, + const char *user_extra_data_central, mz_uint user_extra_data_central_len); + +#ifndef MINIZ_NO_STDIO + /* Adds the contents of a disk file to an archive. This function also records the disk file's modified time into the archive. */ + /* level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION. */ + MINIZ_EXPORT mz_bool mz_zip_writer_add_file(mz_zip_archive *pZip, const char *pArchive_name, const char *pSrc_filename, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags); + + /* Like mz_zip_writer_add_file(), except the file data is read from the specified FILE stream. */ + MINIZ_EXPORT mz_bool mz_zip_writer_add_cfile(mz_zip_archive *pZip, const char *pArchive_name, MZ_FILE *pSrc_file, mz_uint64 max_size, + const MZ_TIME_T *pFile_time, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, const char *user_extra_data_local, mz_uint user_extra_data_local_len, + const char *user_extra_data_central, mz_uint user_extra_data_central_len); +#endif + + /* Adds a file to an archive by fully cloning the data from another archive. */ + /* This function fully clones the source file's compressed data (no recompression), along with its full filename, extra data (it may add or modify the zip64 local header extra data field), and the optional descriptor following the compressed data. */ + MINIZ_EXPORT mz_bool mz_zip_writer_add_from_zip_reader(mz_zip_archive *pZip, mz_zip_archive *pSource_zip, mz_uint src_file_index); + + /* Finalizes the archive by writing the central directory records followed by the end of central directory record. */ + /* After an archive is finalized, the only valid call on the mz_zip_archive struct is mz_zip_writer_end(). */ + /* An archive must be manually finalized by calling this function for it to be valid. */ + MINIZ_EXPORT mz_bool mz_zip_writer_finalize_archive(mz_zip_archive *pZip); + + /* Finalizes a heap archive, returning a pointer to the heap block and its size. */ + /* The heap block will be allocated using the mz_zip_archive's alloc/realloc callbacks. */ + MINIZ_EXPORT mz_bool mz_zip_writer_finalize_heap_archive(mz_zip_archive *pZip, void **ppBuf, size_t *pSize); + + /* Ends archive writing, freeing all allocations, and closing the output file if mz_zip_writer_init_file() was used. */ + /* Note for the archive to be valid, it *must* have been finalized before ending (this function will not do it for you). */ + MINIZ_EXPORT mz_bool mz_zip_writer_end(mz_zip_archive *pZip); + + /* -------- Misc. high-level helper functions: */ + + /* mz_zip_add_mem_to_archive_file_in_place() efficiently (but not atomically) appends a memory blob to a ZIP archive. */ + /* Note this is NOT a fully safe operation. If it crashes or dies in some way your archive can be left in a screwed up state (without a central directory). */ + /* level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION. */ + /* TODO: Perhaps add an option to leave the existing central dir in place in case the add dies? We could then truncate the file (so the old central dir would be at the end) if something goes wrong. */ + MINIZ_EXPORT mz_bool mz_zip_add_mem_to_archive_file_in_place(const char *pZip_filename, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags); + MINIZ_EXPORT mz_bool mz_zip_add_mem_to_archive_file_in_place_v2(const char *pZip_filename, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, mz_zip_error *pErr); + +#ifndef MINIZ_NO_STDIO + /* Reads a single file from an archive into a heap block. */ + /* If pComment is not NULL, only the file with the specified comment will be extracted. */ + /* Returns NULL on failure. */ + MINIZ_EXPORT void *mz_zip_extract_archive_file_to_heap(const char *pZip_filename, const char *pArchive_name, size_t *pSize, mz_uint flags); + MINIZ_EXPORT void *mz_zip_extract_archive_file_to_heap_v2(const char *pZip_filename, const char *pArchive_name, const char *pComment, size_t *pSize, mz_uint flags, mz_zip_error *pErr); +#endif + +#endif /* #ifndef MINIZ_NO_ARCHIVE_WRITING_APIS */ + +#ifdef __cplusplus +} +#endif + +#endif /* MINIZ_NO_ARCHIVE_APIS */ diff --git a/src/helpers/nrf52/DogechatBLEService.cpp b/src/helpers/nrf52/DogechatBLEService.cpp new file mode 100644 index 000000000..d3b14e8c2 --- /dev/null +++ b/src/helpers/nrf52/DogechatBLEService.cpp @@ -0,0 +1,397 @@ +#include "DogechatBLEService.h" + +#ifdef NRF52_PLATFORM + +#include + +// HardFault handler - catches crashes and prints debug info +extern "C" { + void HardFault_Handler(void) { + Serial.println("\n\n!!! HARDFAULT DETECTED !!!"); + Serial.println("Crash - likely stack overflow or memory corruption"); + Serial.flush(); + + // Blink LED rapidly to indicate crash + #ifdef LED_BUILTIN + pinMode(LED_BUILTIN, OUTPUT); + #endif + + while(1) { + #ifdef LED_BUILTIN + digitalWrite(LED_BUILTIN, HIGH); + delay(100); + digitalWrite(LED_BUILTIN, LOW); + delay(100); + #else + delay(200); + #endif + } + } +} + +// Debug output - Adafruit nRF52 core supports Serial.printf +#if DOGECHAT_DEBUG + #define DOGECHAT_DEBUG_PRINTLN(...) do { Serial.printf("DOGECHAT_BLE: "); Serial.printf(__VA_ARGS__); Serial.println(); } while(0) +#else + #define DOGECHAT_DEBUG_PRINTLN(...) {} +#endif + +// Dogechat service UUID: F47B5E2D-4A9E-4C5A-9B3F-8E1D2C3A4B5C +// Bluefruit uses little-endian byte order for UUIDs +static const uint8_t DOGECHAT_SERVICE_UUID_BYTES[] = { + 0x5C, 0x4B, 0x3A, 0x2C, 0x1D, 0x8E, 0x3F, 0x9B, + 0x5A, 0x4C, 0x9E, 0x4A, 0x2D, 0x5E, 0x7B, 0xF4 +}; + +// Dogechat characteristic UUID: A1B2C3D4-E5F6-4A5B-8C9D-0E1F2A3B4C5D +// (Same as ESP32 DOGECHAT_CHARACTERISTIC_UUID in DogechatProtocol.h) +// Bluefruit uses little-endian byte order for UUIDs +static const uint8_t DOGECHAT_CHARACTERISTIC_UUID_BYTES[] = { + 0x5D, 0x4C, 0x3B, 0x2A, 0x1F, 0x0E, 0x9D, 0x8C, + 0x5B, 0x4A, 0xF6, 0xE5, 0xD4, 0xC3, 0xB2, 0xA1 +}; + +// Singleton instance for static callback access +DogechatBLEService* DogechatBLEService::_instance = nullptr; + +DogechatBLEService::DogechatBLEService() + : _service(DOGECHAT_SERVICE_UUID_BYTES) + , _characteristic(DOGECHAT_CHARACTERISTIC_UUID_BYTES) + , _callback(nullptr) + , _serviceActive(false) + , _dogechatClientCount(0) + , _clientSubscribed(false) + , _pendingConnect(false) + , _pendingData(false) + , _writeBufferOffset(0) + , _lastWriteTime(0) + , _queueHead(0) + , _queueTail(0) + , _hasPendingOutgoing(false) +{ + memset(_writeBuffer, 0, sizeof(_writeBuffer)); + memset(_deviceName, 0, sizeof(_deviceName)); + strcpy(_deviceName, "Dogechat"); + for (size_t i = 0; i < MESSAGE_QUEUE_SIZE; i++) { + _messageQueue[i].valid = false; + } + memset(&_pendingOutgoing, 0, sizeof(_pendingOutgoing)); + _instance = this; +} + +bool DogechatBLEService::beginStandalone(const char* deviceName, DogechatBLECallback* callback) { + if (callback == nullptr) { + return false; + } + + _callback = callback; + strncpy(_deviceName, deviceName, sizeof(_deviceName) - 1); + _deviceName[sizeof(_deviceName) - 1] = '\0'; + + // Configure connection parameters BEFORE begin() - set MTU to 517 (max BLE MTU) + // This matches ESP32's BLEDevice::setMTU(517) + // Parameters: mtu_max, event_len, hvn_qsize, wrcmd_qsize + Bluefruit.configPrphConn(517, BLE_GAP_EVENT_LENGTH_DEFAULT, BLE_GATTS_HVN_TX_QUEUE_SIZE_DEFAULT, BLE_GATTC_WRITE_CMD_TX_QUEUE_SIZE_DEFAULT); + + // Initialize Bluefruit + Bluefruit.begin(); + Bluefruit.setTxPower(4); // Max power for better range + + // Set up connection callbacks + Bluefruit.Periph.setConnectCallback(onConnect); + Bluefruit.Periph.setDisconnectCallback(onDisconnect); + + // Dogechat uses open security (no PIN required) + Bluefruit.Security.setMITM(false); + Bluefruit.Security.setIOCaps(false, false, false); + + // Set device name (filter out non-ASCII characters for BLE) + char safeName[32]; + size_t j = 0; + for (size_t i = 0; deviceName[i] != '\0' && j < sizeof(safeName) - 1; i++) { + if (deviceName[i] >= 0x20 && deviceName[i] <= 0x7E) { + safeName[j++] = deviceName[i]; + } + } + safeName[j] = '\0'; + if (j == 0) strcpy(safeName, "Dogechat"); + + Bluefruit.setName(safeName); + + // Configure the Dogechat service + _service.begin(); + + // Configure the characteristic with READ, WRITE, WRITE_NR, NOTIFY properties + _characteristic.setProperties(CHR_PROPS_READ | CHR_PROPS_WRITE | CHR_PROPS_WRITE_WO_RESP | CHR_PROPS_NOTIFY | CHR_PROPS_INDICATE); + _characteristic.setPermission(SECMODE_OPEN, SECMODE_OPEN); // Open security + _characteristic.setMaxLen(512); // Support large Dogechat messages + _characteristic.setWriteCallback(onCharacteristicWrite); + _characteristic.setCccdWriteCallback(onCharacteristicCccdWrite); + _characteristic.begin(); + + _serviceActive = true; + DOGECHAT_DEBUG_PRINTLN("Dogechat BLE service initialized: %s", safeName); + + return true; +} + +void DogechatBLEService::startAdvertising() { + // Clear any previous advertising data + Bluefruit.Advertising.clearData(); + Bluefruit.ScanResponse.clearData(); + + // Set Dogechat UUID in MAIN advertisement (required for Dogechat app discovery) + // The Dogechat Android app filters on service UUID in main advertisement packet + Bluefruit.Advertising.addFlags(BLE_GAP_ADV_FLAGS_LE_ONLY_GENERAL_DISC_MODE); + Bluefruit.Advertising.addService(_service); + + // Put device name in scan response (not main adv - no room with 128-bit UUID) + Bluefruit.ScanResponse.addName(); + + // Configure advertising parameters + Bluefruit.Advertising.restartOnDisconnect(true); + Bluefruit.Advertising.setInterval(32, 244); // in units of 0.625 ms + Bluefruit.Advertising.setFastTimeout(30); // seconds in fast mode + Bluefruit.Advertising.start(0); // 0 = Don't stop advertising + + DOGECHAT_DEBUG_PRINTLN("BLE advertising started"); +} + +void DogechatBLEService::onServerDisconnect() { + if (_dogechatClientCount > 0) { + _dogechatClientCount--; + } + + if (_dogechatClientCount == 0) { + _clientSubscribed = false; + clearWriteBuffer(); + if (_callback != nullptr) { + _callback->onDogechatClientDisconnect(); + } + } +} + +void DogechatBLEService::clearWriteBuffer() { + _writeBufferOffset = 0; + memset(_writeBuffer, 0, sizeof(_writeBuffer)); +} + +bool DogechatBLEService::queueMessage(const DogechatMessage& msg) { + size_t nextTail = (_queueTail + 1) % MESSAGE_QUEUE_SIZE; + + if (nextTail == _queueHead) { + DOGECHAT_DEBUG_PRINTLN("Message queue full, dropping message"); + return false; + } + + _messageQueue[_queueTail].msg = msg; + _messageQueue[_queueTail].valid = true; + _queueTail = nextTail; + + return true; +} + +void DogechatBLEService::processQueue() { + while (_queueHead != _queueTail) { + if (_messageQueue[_queueHead].valid) { + _messageQueue[_queueHead].valid = false; + + if (_callback != nullptr) { + Serial.println("BLE_SERVICE: processQueue() calling callback..."); + _callback->onDogechatMessageReceived(_messageQueue[_queueHead].msg); + Serial.println("BLE_SERVICE: processQueue() callback returned"); + } + } + _queueHead = (_queueHead + 1) % MESSAGE_QUEUE_SIZE; + } +} + +void DogechatBLEService::loop() { + uint32_t now = millis(); + + // Handle deferred connect callback + if (_pendingConnect) { + _pendingConnect = false; + if (_callback != nullptr) { + _callback->onDogechatClientConnect(); + } + } + + // Handle deferred data processing + // Wait 100ms after last write before processing to allow multi-chunk messages to arrive + if (_pendingData && (now - _lastWriteTime >= 100)) { + _pendingData = false; + DOGECHAT_DEBUG_PRINTLN("Processing %u buffered bytes, first bytes: %02X %02X %02X %02X", + (unsigned)_writeBufferOffset, + _writeBufferOffset > 0 ? _writeBuffer[0] : 0, + _writeBufferOffset > 1 ? _writeBuffer[1] : 0, + _writeBufferOffset > 2 ? _writeBuffer[2] : 0, + _writeBufferOffset > 3 ? _writeBuffer[3] : 0); + + DogechatMessage msg; + if (DogechatProtocol::parseMessage(_writeBuffer, _writeBufferOffset, msg)) { + if (DogechatProtocol::validateMessage(msg)) { + DOGECHAT_DEBUG_PRINTLN("Received Dogechat message: type=%02X, len=%d", msg.type, msg.payloadLength); + queueMessage(msg); + } else { + DOGECHAT_DEBUG_PRINTLN("Invalid Dogechat message received (validation failed)"); + } + clearWriteBuffer(); + } else { + DOGECHAT_DEBUG_PRINTLN("Parse failed, have %u bytes, need more or invalid data", (unsigned)_writeBufferOffset); + if (_writeBufferOffset >= DOGECHAT_HEADER_SIZE) { + size_t expectedMin = DogechatProtocol::getMessageSize(msg); + DOGECHAT_DEBUG_PRINTLN("Expected min size: %u", (unsigned)expectedMin); + if (_writeBufferOffset > expectedMin + 100) { + DOGECHAT_DEBUG_PRINTLN("Write buffer contains unparseable data, clearing"); + clearWriteBuffer(); + } + } + } + } + + // Check for write buffer timeout + if (_writeBufferOffset > 0) { + if (now - _lastWriteTime > WRITE_TIMEOUT_MS) { + DOGECHAT_DEBUG_PRINTLN("Write buffer timeout, clearing"); + clearWriteBuffer(); + } + } + + // Process queued messages + processQueue(); + + // Debug: mark end of loop iteration if we processed something + static uint32_t lastLoopPrint = 0; + if (now - lastLoopPrint > 5000) { + Serial.println("BLE_SERVICE: loop() heartbeat"); + lastLoopPrint = now; + } +} + +bool DogechatBLEService::broadcastMessage(const DogechatMessage& msg) { + Serial.println("BLE_SERVICE: >>> broadcastMessage() ENTRY <<<"); + DOGECHAT_DEBUG_PRINTLN("broadcastMessage: type=0x%02X, active=%d, subscribed=%d, clients=%d", + msg.type, _serviceActive, _clientSubscribed, _dogechatClientCount); + + if (!_serviceActive) { + DOGECHAT_DEBUG_PRINTLN("broadcastMessage: service not active"); + return false; + } + + // Serialize message - use static buffer to avoid stack overflow + static uint8_t buffer[DOGECHAT_MAX_MESSAGE_SIZE]; + size_t len = DogechatProtocol::serializeMessage(msg, buffer, sizeof(buffer)); + if (len == 0) { + DOGECHAT_DEBUG_PRINTLN("broadcastMessage: serialize failed"); + return false; + } + + // Always set the characteristic value so it can be read + _characteristic.write(buffer, len); + DOGECHAT_DEBUG_PRINTLN("broadcastMessage: set characteristic value (%u bytes)", (unsigned)len); + + // If client connected but not yet subscribed, queue for notification later + if (_dogechatClientCount > 0 && !_clientSubscribed) { + DOGECHAT_DEBUG_PRINTLN("broadcastMessage: client not subscribed, queuing notify for later"); + _pendingOutgoing = msg; + _hasPendingOutgoing = true; + return true; // Value is set for reading, notify will happen when they subscribe + } + + // Try to send via notify if we have subscribers + if (_clientSubscribed) { + Serial.println("BLE_SERVICE: Client subscribed, sending notify"); + uint16_t result = _characteristic.notify(buffer, len); + DOGECHAT_DEBUG_PRINTLN("broadcastMessage: notify returned %u", result); + if (result) { + DOGECHAT_DEBUG_PRINTLN("TX: type=0x%02X, len=%u", msg.type, (unsigned)len); + } else { + Serial.println("BLE_SERVICE: WARNING - notify returned 0 (failed)"); + } + } else { + Serial.println("BLE_SERVICE: Client NOT subscribed, only set characteristic value"); + } + + return true; +} + +void DogechatBLEService::sendPendingOutgoing() { + if (!_hasPendingOutgoing || !_clientSubscribed) { + return; + } + + DOGECHAT_DEBUG_PRINTLN("Sending pending outgoing message"); + _hasPendingOutgoing = false; + + // Serialize and send the pending message - use static buffer to avoid stack overflow + static uint8_t buffer[DOGECHAT_MAX_MESSAGE_SIZE]; + size_t len = DogechatProtocol::serializeMessage(_pendingOutgoing, buffer, sizeof(buffer)); + if (len > 0) { + uint16_t result = _characteristic.notify(buffer, len); + DOGECHAT_DEBUG_PRINTLN("Pending message notify returned %u", result); + } +} + +// Static callbacks + +void DogechatBLEService::onConnect(uint16_t conn_handle) { + if (_instance != nullptr) { + _instance->_dogechatClientCount++; + _instance->_pendingConnect = true; + DOGECHAT_DEBUG_PRINTLN("BLE client connected"); + } +} + +void DogechatBLEService::onDisconnect(uint16_t conn_handle, uint8_t reason) { + if (_instance != nullptr) { + _instance->onServerDisconnect(); + DOGECHAT_DEBUG_PRINTLN("BLE client disconnected, reason=0x%02X", reason); + } +} + +void DogechatBLEService::onCharacteristicWrite(uint16_t conn_handle, BLECharacteristic* chr, uint8_t* data, uint16_t len) { + DOGECHAT_DEBUG_PRINTLN("BLE WRITE received: %u bytes", len); + + if (_instance == nullptr || len == 0) { + return; + } + + _instance->_lastWriteTime = millis(); + _instance->_pendingData = true; + + // Append to write buffer + size_t copyLen = len; + if (_instance->_writeBufferOffset + copyLen > sizeof(_instance->_writeBuffer)) { + DOGECHAT_DEBUG_PRINTLN("Write buffer overflow, clearing"); + _instance->clearWriteBuffer(); + copyLen = (len > sizeof(_instance->_writeBuffer)) ? sizeof(_instance->_writeBuffer) : len; + } + + memcpy(&_instance->_writeBuffer[_instance->_writeBufferOffset], data, copyLen); + _instance->_writeBufferOffset += copyLen; + DOGECHAT_DEBUG_PRINTLN("Write buffer now has %u bytes", (unsigned)_instance->_writeBufferOffset); +} + +void DogechatBLEService::onCharacteristicCccdWrite(uint16_t conn_handle, BLECharacteristic* chr, uint16_t cccd_value) { + Serial.print("BLE_SERVICE: CCCD write callback, cccd_value=0x"); + Serial.println(cccd_value, HEX); + + if (_instance != nullptr) { + bool wasSubscribed = _instance->_clientSubscribed; + _instance->_clientSubscribed = (cccd_value & BLE_GATT_HVX_NOTIFICATION) != 0; + DOGECHAT_DEBUG_PRINTLN("CCCD write: notifications %s (was %s)", + _instance->_clientSubscribed ? "enabled" : "disabled", + wasSubscribed ? "enabled" : "disabled"); + Serial.print("BLE_SERVICE: _clientSubscribed now = "); + Serial.println(_instance->_clientSubscribed ? "true" : "false"); + + // If client just subscribed and we have pending messages, send them + if (!wasSubscribed && _instance->_clientSubscribed) { + Serial.println("BLE_SERVICE: Client just subscribed, sending pending"); + _instance->sendPendingOutgoing(); + } + } +} + +#endif // NRF52_PLATFORM diff --git a/src/helpers/nrf52/DogechatBLEService.h b/src/helpers/nrf52/DogechatBLEService.h new file mode 100644 index 000000000..b538f357c --- /dev/null +++ b/src/helpers/nrf52/DogechatBLEService.h @@ -0,0 +1,136 @@ +#pragma once + +#ifdef NRF52_PLATFORM + +#include "../dogechat/DogechatProtocol.h" +#include + +/** + * Callback interface for Dogechat BLE events + */ +class DogechatBLECallback { +public: + virtual ~DogechatBLECallback() {} + + /** + * Called when a Dogechat message is received via BLE + * @param msg The received message + */ + virtual void onDogechatMessageReceived(const DogechatMessage& msg) = 0; + + /** + * Called when a Dogechat BLE client connects + */ + virtual void onDogechatClientConnect() {} + + /** + * Called when a Dogechat BLE client disconnects + */ + virtual void onDogechatClientDisconnect() {} +}; + +/** + * Dogechat BLE Service for NRF52 (using Bluefruit) + * Provides a GATT service for Dogechat protocol communication + * + * Note: This uses standalone BLE advertising. For NRF52, when Dogechat is enabled, + * the MeshCore companion uses USB serial while Dogechat uses BLE independently. + */ +class DogechatBLEService { +public: + DogechatBLEService(); + + /** + * Initialize BLE and start the Dogechat service in standalone mode + * Creates BLE server with Dogechat service only. + * @param deviceName BLE device name for advertising + * @param callback Callback for Dogechat events + * @return true if successful + */ + bool beginStandalone(const char* deviceName, DogechatBLECallback* callback); + + /** + * Start BLE advertising + * Call after beginStandalone() to start advertising + */ + void startAdvertising(); + + /** + * Check if the service is active + */ + bool isActive() const { return _serviceActive; } + + /** + * Check if a Dogechat client is connected + */ + bool hasConnectedClient() const { return _dogechatClientCount > 0; } + + /** + * Broadcast a message to connected Dogechat clients + * @param msg Message to send + * @return true if message was sent + */ + bool broadcastMessage(const DogechatMessage& msg); + + /** + * Process loop - call from main loop + * Handles deferred message processing + */ + void loop(); + + /** + * Called when a client disconnects + */ + void onServerDisconnect(); + +private: + // Bluefruit service and characteristic + BLEService _service; + BLECharacteristic _characteristic; + DogechatBLECallback* _callback; + char _deviceName[48]; + + bool _serviceActive; + uint8_t _dogechatClientCount; + bool _clientSubscribed; + + // Flags for deferred processing + volatile bool _pendingConnect; + volatile bool _pendingData; + + // Write buffer for reassembling fragmented BLE writes + uint8_t _writeBuffer[512]; + size_t _writeBufferOffset; + uint32_t _lastWriteTime; + static const uint32_t WRITE_TIMEOUT_MS = 5000; + + // Message queue for deferred processing (incoming) + static const size_t MESSAGE_QUEUE_SIZE = 8; + struct QueuedMessage { + DogechatMessage msg; + bool valid; + }; + QueuedMessage _messageQueue[MESSAGE_QUEUE_SIZE]; + volatile size_t _queueHead; + volatile size_t _queueTail; + + // Pending outgoing message (when client not yet subscribed) + DogechatMessage _pendingOutgoing; + bool _hasPendingOutgoing; + + void clearWriteBuffer(); + bool queueMessage(const DogechatMessage& msg); + void processQueue(); + void sendPendingOutgoing(); + + // Static callbacks for Bluefruit (will call instance methods) + static void onConnect(uint16_t conn_handle); + static void onDisconnect(uint16_t conn_handle, uint8_t reason); + static void onCharacteristicWrite(uint16_t conn_handle, BLECharacteristic* chr, uint8_t* data, uint16_t len); + static void onCharacteristicCccdWrite(uint16_t conn_handle, BLECharacteristic* chr, uint16_t cccd_value); + + // Singleton for static callback access + static DogechatBLEService* _instance; +}; + +#endif // NRF52_PLATFORM diff --git a/variants/ebyte_eora_s3/platformio.ini b/variants/ebyte_eora_s3/platformio.ini index bdf6bba3d..3444f9147 100644 --- a/variants/ebyte_eora_s3/platformio.ini +++ b/variants/ebyte_eora_s3/platformio.ini @@ -134,3 +134,23 @@ build_src_filter = ${Ebyte_EoRa-S3.build_src_filter} lib_deps = ${Ebyte_EoRa-S3.lib_deps} densaugeo/base64 @ ~1.4.0 + +; USB Serial Companion with standalone Dogechat BLE +[env:Ebyte_EoRa-S3_companion_radio_usb_dogechat] +extends = Ebyte_EoRa-S3 +build_flags = + ${Ebyte_EoRa-S3.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D DISPLAY_CLASS=SSD1306Display + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 +build_src_filter = ${Ebyte_EoRa-S3.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Ebyte_EoRa-S3.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/heltec_ct62/platformio.ini b/variants/heltec_ct62/platformio.ini index 1f2e330a3..715d9be7a 100644 --- a/variants/heltec_ct62/platformio.ini +++ b/variants/heltec_ct62/platformio.ini @@ -149,3 +149,19 @@ build_src_filter = ${Heltec_ct62.build_src_filter} lib_deps = ${Heltec_ct62.lib_deps} ${esp32_ota.lib_deps} + +; USB Serial Companion with standalone Dogechat BLE +[env:Heltec_ct62_companion_radio_usb_dogechat] +extends = Heltec_ct62 +build_flags = + ${Heltec_ct62.build_flags} + ${dogechat_base.build_flags} + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D OFFLINE_QUEUE_SIZE=256 +build_src_filter = ${Heltec_ct62.build_src_filter} + ${dogechat_base.build_src_filter} + +<../examples/companion_radio/*.cpp> +lib_deps = + ${Heltec_ct62.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/heltec_tracker_v2/platformio.ini b/variants/heltec_tracker_v2/platformio.ini index 61ccd4f4b..95a59e55c 100644 --- a/variants/heltec_tracker_v2/platformio.ini +++ b/variants/heltec_tracker_v2/platformio.ini @@ -216,3 +216,23 @@ build_src_filter = ${Heltec_tracker_v2.build_src_filter} lib_deps = ${Heltec_tracker_v2.lib_deps} ${esp32_ota.lib_deps} + +; USB Serial Companion with standalone Dogechat BLE +[env:heltec_tracker_v2_companion_radio_usb_dogechat] +extends = Heltec_tracker_v2 +build_flags = + ${Heltec_tracker_v2.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D DISPLAY_CLASS=ST7735Display +build_src_filter = ${Heltec_tracker_v2.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Heltec_tracker_v2.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/heltec_v2/platformio.ini b/variants/heltec_v2/platformio.ini index 539cc52e8..46035a0b5 100644 --- a/variants/heltec_v2/platformio.ini +++ b/variants/heltec_v2/platformio.ini @@ -194,3 +194,23 @@ build_src_filter = ${Heltec_lora32_v2.build_src_filter} lib_deps = ${Heltec_lora32_v2.lib_deps} densaugeo/base64 @ ~1.4.0 + +; USB Serial Companion with standalone Dogechat BLE +[env:Heltec_v2_companion_radio_usb_dogechat] +extends = Heltec_lora32_v2 +build_flags = + ${Heltec_lora32_v2.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D DISPLAY_CLASS=SSD1306Display + -D MAX_CONTACTS=160 + -D MAX_GROUP_CHANNELS=8 +build_src_filter = ${Heltec_lora32_v2.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Heltec_lora32_v2.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/heltec_v3/platformio.ini b/variants/heltec_v3/platformio.ini index 36c6386f6..2bf4b959a 100644 --- a/variants/heltec_v3/platformio.ini +++ b/variants/heltec_v3/platformio.ini @@ -332,6 +332,44 @@ lib_deps = ${Heltec_lora32_v3.lib_deps} densaugeo/base64 @ ~1.4.0 +; USB Serial Companion with standalone Dogechat BLE +; MeshCore companion app uses USB serial, Dogechat app uses BLE +[env:Heltec_WSL3_companion_radio_usb_dogechat] +extends = Heltec_lora32_v3 +build_flags = + ${Heltec_lora32_v3.build_flags} + ${dogechat_base.build_flags} + -D MAX_CONTACTS=140 + -D MAX_GROUP_CHANNELS=40 + -D DOGECHAT_DEBUG=1 + -D DOGECHAT_DEBUG_PACKETDUMP=1 +build_src_filter = ${Heltec_lora32_v3.build_src_filter} + ${dogechat_base.build_src_filter} + +<../examples/companion_radio/*.cpp> +lib_deps = + ${Heltec_lora32_v3.lib_deps} + densaugeo/base64 @ ~1.4.0 + +; USB Serial Companion with Dogechat BLE (V3 with display) +[env:Heltec_v3_companion_radio_usb_dogechat] +extends = Heltec_lora32_v3 +build_flags = + ${Heltec_lora32_v3.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D DISPLAY_CLASS=SSD1306Display +build_src_filter = ${Heltec_lora32_v3.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Heltec_lora32_v3.lib_deps} + densaugeo/base64 @ ~1.4.0 + [env:Heltec_WSL3_companion_radio_wifi] extends = Heltec_lora32_v3 build_flags = diff --git a/variants/heltec_v4/platformio.ini b/variants/heltec_v4/platformio.ini index c26a5bc69..63d1bc7f7 100644 --- a/variants/heltec_v4/platformio.ini +++ b/variants/heltec_v4/platformio.ini @@ -207,3 +207,25 @@ build_src_filter = ${Heltec_lora32_v4.build_src_filter} lib_deps = ${Heltec_lora32_v4.lib_deps} ${esp32_ota.lib_deps} + +; USB Serial Companion with Dogechat BLE (V4 with display) +[env:heltec_v4_companion_radio_usb_dogechat] +extends = Heltec_lora32_v4 +build_flags = + ${Heltec_lora32_v4.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D DISPLAY_CLASS=SSD1306Display + -D DOGECHAT_DEBUG=1 + -D DOGECHAT_DEBUG_PACKETDUMP=1 +build_src_filter = ${Heltec_lora32_v4.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Heltec_lora32_v4.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/lilygo_t3s3/platformio.ini b/variants/lilygo_t3s3/platformio.ini index 0f01c9b76..2e6f9a669 100644 --- a/variants/lilygo_t3s3/platformio.ini +++ b/variants/lilygo_t3s3/platformio.ini @@ -172,3 +172,23 @@ build_src_filter = ${LilyGo_T3S3_sx1262.build_src_filter} lib_deps = ${LilyGo_T3S3_sx1262.lib_deps} densaugeo/base64 @ ~1.4.0 + +; USB Serial Companion with standalone Dogechat BLE +[env:LilyGo_T3S3_sx1262_companion_radio_usb_dogechat] +extends = LilyGo_T3S3_sx1262 +build_flags = + ${LilyGo_T3S3_sx1262.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D DISPLAY_CLASS=SSD1306Display + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 +build_src_filter = ${LilyGo_T3S3_sx1262.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${LilyGo_T3S3_sx1262.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/lilygo_tdeck/platformio.ini b/variants/lilygo_tdeck/platformio.ini index adfe9d1e3..46b42f0e6 100644 --- a/variants/lilygo_tdeck/platformio.ini +++ b/variants/lilygo_tdeck/platformio.ini @@ -93,4 +93,23 @@ build_src_filter = ${LilyGo_TDeck.build_src_filter} + lib_deps = ${LilyGo_TDeck.lib_deps} - ${esp32_ota.lib_deps} \ No newline at end of file + ${esp32_ota.lib_deps} +; USB Serial Companion with standalone Dogechat BLE +[env:LilyGo_TDeck_companion_radio_usb_dogechat] +extends = LilyGo_TDeck +build_flags = + ${LilyGo_TDeck.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D OFFLINE_QUEUE_SIZE=256 +build_src_filter = ${LilyGo_TDeck.build_src_filter} + ${dogechat_base.build_src_filter} + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> + + +lib_deps = + ${LilyGo_TDeck.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/lilygo_tlora_v2_1/platformio.ini b/variants/lilygo_tlora_v2_1/platformio.ini index 9ef9734e5..eae328778 100644 --- a/variants/lilygo_tlora_v2_1/platformio.ini +++ b/variants/lilygo_tlora_v2_1/platformio.ini @@ -108,6 +108,24 @@ lib_deps = ${LilyGo_TLora_V2_1_1_6.lib_deps} densaugeo/base64 @ ~1.4.0 +; USB Serial Companion with standalone Dogechat BLE +[env:LilyGo_TLora_V2_1_1_6_companion_radio_usb_dogechat] +extends = LilyGo_TLora_V2_1_1_6 +build_flags = + ${LilyGo_TLora_V2_1_1_6.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=160 + -D MAX_GROUP_CHANNELS=8 +build_src_filter = ${LilyGo_TLora_V2_1_1_6.build_src_filter} + ${dogechat_base.build_src_filter} + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${LilyGo_TLora_V2_1_1_6.lib_deps} + densaugeo/base64 @ ~1.4.0 + [env:LilyGo_TLora_V2_1_1_6_room_server] extends = LilyGo_TLora_V2_1_1_6 build_src_filter = ${LilyGo_TLora_V2_1_1_6.build_src_filter} @@ -190,4 +208,5 @@ build_flags = ; -D CORE_DEBUG_LEVEL=3 lib_deps = ${LilyGo_TLora_V2_1_1_6.lib_deps} - ${esp32_ota.lib_deps} \ No newline at end of file + ${esp32_ota.lib_deps} + diff --git a/variants/station_g2/platformio.ini b/variants/station_g2/platformio.ini index b3ba2a863..8881e1201 100644 --- a/variants/station_g2/platformio.ini +++ b/variants/station_g2/platformio.ini @@ -235,3 +235,20 @@ build_src_filter = ${Station_G2.build_src_filter} lib_deps = ${Station_G2.lib_deps} densaugeo/base64 @ ~1.4.0 + +; USB Serial Companion with standalone Dogechat BLE +[env:Station_G2_companion_radio_usb_dogechat] +extends = Station_G2 +build_flags = + ${Station_G2.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 +build_src_filter = ${Station_G2.build_src_filter} + ${dogechat_base.build_src_filter} + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Station_G2.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/wio-tracker-l1/platformio.ini b/variants/wio-tracker-l1/platformio.ini index 31c6bcb07..5b4245516 100644 --- a/variants/wio-tracker-l1/platformio.ini +++ b/variants/wio-tracker-l1/platformio.ini @@ -107,3 +107,33 @@ lib_deps = ${WioTrackerL1.lib_deps} adafruit/RTClib @ ^2.1.3 densaugeo/base64 @ ~1.4.0 end2endzone/NonBlockingRTTTL@^1.3.0 + +; USB serial for MeshCore companion + standalone BLE for Dogechat +[env:WioTrackerL1_companion_radio_usb_dogechat] +extends = WioTrackerL1 +board_build.ldscript = boards/nrf52840_s140_v7_extrafs.ld +board_upload.maximum_size = 708608 +build_flags = ${WioTrackerL1.build_flags} + ${dogechat_nrf52.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D DISPLAY_CLASS=SH1106Display + -D UI_HAS_JOYSTICK=1 + -D OFFLINE_QUEUE_SIZE=256 + -D PIN_BUZZER=12 + -D QSPIFLASH=1 + -D DOGECHAT_DEBUG=1 + -D DOGECHAT_DEBUG_PACKETDUMP=1 + -D MESH_DEBUG=1 +build_src_filter = ${WioTrackerL1.build_src_filter} + ${dogechat_nrf52.build_src_filter} + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> + + + + +lib_deps = ${WioTrackerL1.lib_deps} + adafruit/RTClib @ ^2.1.3 + densaugeo/base64 @ ~1.4.0 + end2endzone/NonBlockingRTTTL@^1.3.0 diff --git a/variants/xiao_c3/platformio.ini b/variants/xiao_c3/platformio.ini index 76b72174a..7b96fd2dd 100644 --- a/variants/xiao_c3/platformio.ini +++ b/variants/xiao_c3/platformio.ini @@ -123,3 +123,19 @@ lib_deps = ${Xiao_esp32_C3.lib_deps} ${esp32_ota.lib_deps} densaugeo/base64 @ ~1.4.0 + +; USB Serial Companion with standalone Dogechat BLE +[env:Xiao_C3_companion_radio_usb_dogechat] +extends = Xiao_esp32_C3 +build_flags = + ${Xiao_esp32_C3.build_flags} + ${dogechat_base.build_flags} + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D OFFLINE_QUEUE_SIZE=256 +build_src_filter = ${Xiao_esp32_C3.build_src_filter} + ${dogechat_base.build_src_filter} + +<../examples/companion_radio/*.cpp> +lib_deps = + ${Xiao_esp32_C3.lib_deps} + densaugeo/base64 @ ~1.4.0 diff --git a/variants/xiao_s3_wio/platformio.ini b/variants/xiao_s3_wio/platformio.ini index 8979edc25..131f5c261 100644 --- a/variants/xiao_s3_wio/platformio.ini +++ b/variants/xiao_s3_wio/platformio.ini @@ -210,3 +210,26 @@ build_flags = lib_deps = ${Xiao_S3_WIO.lib_deps} ${esp32_ota.lib_deps} + +; USB Serial Companion with standalone Dogechat BLE +[env:Xiao_S3_WIO_companion_radio_usb_dogechat] +extends = Xiao_S3_WIO +build_flags = + ${Xiao_S3_WIO.build_flags} + ${dogechat_base.build_flags} + -I examples/companion_radio/ui-new + -D MAX_CONTACTS=350 + -D MAX_GROUP_CHANNELS=40 + -D DISPLAY_CLASS=SSD1306Display + -D OFFLINE_QUEUE_SIZE=256 + -D LED_STATE_ON=LOW +build_src_filter = ${Xiao_S3_WIO.build_src_filter} + ${dogechat_base.build_src_filter} + + + + + +<../examples/companion_radio/*.cpp> + +<../examples/companion_radio/ui-new/*.cpp> +lib_deps = + ${Xiao_S3_WIO.lib_deps} + densaugeo/base64 @ ~1.4.0 + adafruit/Adafruit SSD1306 @ ^2.5.13