mozzwald/fujinet-firmware
1
0
Fork 0
mirror of https://github.com/FujiNetWIFI/fujinet-firmware.git synced 2026-09-17 02:57:35 -05:00
Code Issues Projects Releases Packages Wiki Activity
8-bit systems to ESP32 WiFi Multifunction Firmware https://fujinet.online
7703 commits 59 branches 27 tags 138 MiB
Find a file
Exact
Thomas Cherryhomes 632e3119c5
 [http] fix OneDrive uploads: size the request TX buffer to fit large headers (#1554) (#1639) 
Fixes #1554.

## What was wrong

Not OneDrive, and not the August regression it looked like.
`ONEDRIVE.cpp` has exactly one commit in its history (`8b05d7dcc`,
2026-07-15) and is byte-identical at the reporter's build and at HEAD —
ESP32 OneDrive has never worked.

The bug is in the vendored HTTP client, `lib/fn_esp_http_client/`:

- `buffer_size_tx` defaults to `DEFAULT_HTTP_BUF_SIZE` = **512**, and no
caller in the repo sets it.
- `http_header_generate_string()` serialises the request headers in
buffer-sized chunks and **will not split a single item**. When one item
alone doesn't fit, it wrote nothing, set `*buffer_len = 0`, and left
`is_end` false — so the blank line terminating the header block was
never emitted. The caller loop then hit `if (wlen <= 0) break;` and
`esp_http_client_request_send()` returned `ESP_OK`.

A Microsoft Graph bearer token is 1.2–2.5 KB, so `Authorization: Bearer
<token>` never fit. The cutoff is a token of ~489 characters; Google's
are usually shorter, which is why GDRIVE mostly works and OneDrive never
did.

Both reported symptoms fall straight out of that:

| Symptom in the log | Mechanism |
|---|---|
| `api_put: HTTP 400`, one second after the SIO close | The upload body
was written into what Graph still read as the header section, so it
rejected immediately. |
| `api_get: HTTP -1`, 15 s and 16 s after the previous line |
`esp_http_client_fetch_headers()` sets `status_code = -1`, then reads
with `client->timeout_ms` (15000 in `api_get`). The server is still
waiting for the rest of the headers, so it times out and the `-1`
sentinel is returned verbatim. |

fujinet-pc was never affected because `mgHttpClient` (mongoose) has no
such buffer.

## The fix

**`lib/fn_esp_http_client/`** — size the request buffer to the longest
single header plus the request line, growing it on demand before the
request line is prepared, while nothing has been transmitted and a
`realloc` is still safe. Sizing is recomputed per request so a
redirect's new path is accounted for, and the buffer only ever grows, so
it survives redirects, auth retries and keep-alive reuse. Past an 8 KB
ceiling, or if the allocation fails, the request is refused rather than
sent malformed.

Memory is pay-per-use: ordinary requests come out under 512 and never
touch the allocator. Only OAuth-bearing calls allocate more — ~2.1 KB
for Graph, for the duration of one request. With
`SPIRAM_MALLOC_ALWAYSINTERNAL` at 4096 that lands in internal DRAM;
routing large grows to PSRAM would be a reasonable follow-up.

The write loop also now aborts rather than sending a header block with
no terminator. That guard needs the new `-1` return rather than a
loop-body check: an oversized header at index 0 returns 0 and leaves the
loop before *anything* is written, not even the request line.

This covers GDRIVE, GMAIL, GCAL, S3 and the OAuth relay calls too, which
all share the 512-byte default, and incidentally fixes the long-URL
`"Out of buffer"` failure in `http_client_prepare_first_line()`.

**`NetworkProtocolFS::close()`** — the second half of the issue. It
computed the error from `close_file()`, used it only to set the status
byte, then returned `FUJI_ERROR::NONE`. `NDevice::fujidev_close()`
*does* check the return value, so the base class was discarding an error
that HTTP, S3, GDRIVE, ONEDRIVE and FTP all deliberately raise. For
protocols that buffer a write and upload at close there is no other
moment to tell the computer the data never landed.

I audited all ten `NetworkProtocolFS` subclasses before changing this.
Eight are safe as-is. Two needed preparing first, in their own commit:

- **HTTP** would have regressed. Its verdict comes from `resultCode`
through `fserror_to_error()`, and `resultCode == 0` falls to the
`default:` arm and becomes `GENERAL`. The GET is deferred until the
first read — `resolve()` sets `resultCode = 0` on purpose — so opening
`N:HTTP://` and closing without reading or polling status would have
started raising a bus error on a perfectly normal close.
- **TNFS** judged on `tnfs_error` even when `fd == 0`, i.e. a stale
error from an earlier call.

**`ONEDRIVE.cpp`** — hardening for the failures the same user would hit
next: log Graph's JSON error body on non-2xx (asked for in the issue — a
failure was previously a bare status number); keep the status in
`_last_http` so a legitimate 2xx with an empty body isn't read as
failure; check `fetch_headers()`; fail on a short write instead of
sending a body shorter than the committed `Content-Length`; and refresh
the token before the close-time PUT, since `ensure_access_token()` only
ran at mount and a long write session could outlive it. The
`mgHttpClient` half gets the same body logging so fujinet-pc stays a
useful reference.

## Testing

`tests/HttpHeaderTests.cpp` pins the chunking boundary on the host —
`fn_http_header.cpp` needs only `esp_log.h` and `esp_err.h`, stubbed
under `tests/esp_stubs/`, so this needs no hardware and CI picks it up
via the existing ctest wiring.

Against the **pre-fix** serialiser two cases fail with exactly the
values seen in the field:

```
ERROR: CHECK( ret == -1 ) is NOT correct!  values: CHECK( 2 == -1 )
ERROR: CHECK( http_header_generate_string(h, 0, &buf[0], &wlen) == -1 ) ... CHECK( 0 == -1 )
```

Builds, all green:

- `./build.sh -b` for `fujinet-atari-v1` and
`fujinet-atari-esp32-s3-wroom-1-n16r8`
- `./build.sh -p ATARI` and `./build.sh -p APPLE` — 6/6 ctest suites
pass
- `./coding-standard.py --against master` clean

Not verified on hardware — I don't have a OneDrive-linked device.
@GLitt-Nostos offered to test and this is the build to try; with `-D
VERBOSE_HTTP` the `Grew request TX buffer 512 -> N` line confirms the
new path fired. Worth exercising a read as well as a write: the download
path goes through `fnHttpClient` rather than the raw client, so it's a
second client instance that shared the same bug.

## Deliberately left out

`close_file_handle()` returns early on an empty buffer, so a zero-byte
file is silently never created — `GDRIVE.cpp` does the same. Separate
issue, not mixed in here.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 18:20:46 -05:00
.github/workflows [ci] build coverage for the pico/ cartridge trees, and let build-board.sh build the fujicard (#1637) 2026-09-16 18:20:29 -05:00
boards [intv] add back in the fujiversal-intv. (#1529) 2026-08-12 15:17:09 -05:00
build-platforms Add fujinet-coco-devkitc-centipede board (#1631) 2026-09-13 07:05:37 -05:00
components Rename FNSGML -> FNHTML, SGMLParser -> HTMLParser (#1642) 2026-09-16 12:38:13 -05:00
components_pc [coco] move transaction_* methods up to virtualDevice (#1408) 2026-06-30 08:57:57 -07:00
data/webui [apple2] restore the Disk ][ slots in the web interface (#1636) 2026-09-15 17:09:52 -05:00
distfiles [apple2] update to support 8 SP devices & updated config image v2 (#1244) 2026-04-13 10:36:16 -07:00
docs Mac68k - Back On Track September (#1633) 2026-09-14 15:37:00 -05:00
include Unify device/network for all platforms. (#1624) 2026-09-09 14:51:16 -05:00
lib [http] fix OneDrive uploads: size the request TX buffer to fit large headers (#1554) (#1639) 2026-09-16 18:20:46 -05:00
mcuconfig Get MSX Fujiversal Proto 260402 working with ESP32 (#1305) 2026-05-19 17:00:34 -07:00
notes Merge branch 'master' 2024-01-12 21:10:21 +01:00
pico [ci] build coverage for the pico/ cartridge trees, and let build-board.sh build the fujicard (#1637) 2026-09-16 18:20:29 -05:00
platformio-ini-files Build config cleanup (#1128) 2026-01-13 19:12:01 -06:00
src Add XML channel mode, and general text sanitization (#1557) 2026-09-16 14:30:32 -05:00
test Merge branch 'master' 2024-01-12 21:10:21 +01:00
tests [http] fix OneDrive uploads: size the request TX buffer to fit large headers (#1554) (#1639) 2026-09-16 18:20:46 -05:00
tools Add onedrive protocol (#1461) 2026-07-15 16:24:54 -05:00
.clang-format Created coding-standard.py to check whitespace/formatting of plain text files (#981) 2025-07-27 16:32:16 -05:00
.dir-locals.el Created coding-standard.py to check whitespace/formatting of plain text files (#981) 2025-07-27 16:32:16 -05:00
.gitignore [intv] add back in the fujiversal-intv. (#1529) 2026-08-12 15:17:09 -05:00
.travis.yml Add jinja2 templating for webUI 2022-12-03 15:47:19 +00:00
bisect-helper.py Make bisect-helper.py able to work with custom names instead of bad/good. (#1370) 2026-06-21 08:57:11 -05:00
build-sh.md [build.sh] move core ini files needed out of root dir so Andy does not delete them. update docs. remove comments 2024-05-27 18:39:11 +01:00
build.sh [adam] AdamNet bus-over-IP transport and ADAM PC target (talk to ADAMEm) (#1379) 2026-06-25 17:05:14 -07:00
build_firmwarezip.py Adds xdrive device support (#1005) 2025-09-02 13:41:49 -05:00
build_packages.py Update build.sh to also do PC builds 2024-01-13 17:21:28 +00:00
build_version.py Update components (#1458) 2026-07-14 22:18:25 -04:00
build_version_pc.py New DaisyChain class to centralize device chain management across buses (#1594) 2026-09-01 08:47:06 -07:00
build_webui.py [apple2] restore the Disk ][ slots in the web interface (#1636) 2026-09-15 17:09:52 -05:00
CMakeLists.txt Applying @fenrock's FujiBusPacket unit tests 2026-06-22 14:22:21 -05:00
coding-standard.py Unification, part 1: Rename fuji device source code to match class name. (#1050) 2025-11-05 10:05:59 -05:00
create-platformio-ini.py Add debug_version.py to put commit ID into FN_VERSION_FULL during build without altering include/version.h 2024-10-16 10:29:30 -04:00
CREDITS.md Small XEX bootloader change... 2020-08-04 15:28:27 -07:00
debug_version.py New DaisyChain class to centralize device chain management across buses (#1594) 2026-09-01 08:47:06 -07:00
fujinet.py Building for Atari 2022-02-02 01:05:36 -06:00
fujinet_firmware_uploader.md [fujinet] added a new python script to upload releases from cli 2024-08-10 18:58:54 -04:00
fujinet_firmware_uploader.py Allow specifying firmware zip on command line (#865) 2025-03-19 22:51:18 -04:00
fujinet_partitions_4MB.csv Update partition tables and mount point in prep for custom OTA functionality (#949) 2025-06-16 09:23:50 -05:00
fujinet_partitions_4MB_noupdate.csv Adds xdrive device support (#1005) 2025-09-02 13:41:49 -05:00
fujinet_partitions_8MB.csv Update partition tables and mount point in prep for custom OTA functionality (#949) 2025-06-16 09:23:50 -05:00
fujinet_partitions_16MB.csv Update partition tables and mount point in prep for custom OTA functionality (#949) 2025-06-16 09:23:50 -05:00
fujinet_pc.cmake Add XML channel mode, and general text sanitization (#1557) 2026-09-16 14:30:32 -05:00
full-verify.sh Consolidate webui configs with inheritance (#1621) 2026-09-09 14:50:59 -05:00
install_python_modules.sh Fixes while investigating mongoose problems with webui in fujinet-pc 2024-03-28 16:08:36 +00:00
LICENSE convert lf to crlf 2023-06-29 11:43:46 -05:00
Makefile Rename BeckerSocket to BoIPChannel and add options to prepare it for Adam LWM. (#1383) 2026-06-25 14:16:43 -05:00
pc_merge_notes.md [PC] re-add get_general_label() 2024-01-08 17:12:44 +01:00
platformio-sample.ini Add Lynx Rev1 ESP32 S3 support (#1271) 2026-04-28 12:40:49 -05:00
python_modules.txt Fixes while investigating mongoose problems with webui in fujinet-pc 2024-03-28 16:08:36 +00:00
README.md fix typos in documentation markdown 2024-05-26 09:17:46 -05:00
README_wifi.md fix typos in documentation markdown 2024-05-26 09:17:46 -05:00
sdkconfig.defaults [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.defaults.psram_octal [intv] add back in the fujiversal-intv. (#1529) 2026-08-12 15:17:09 -05:00
sdkconfig.fujiapple-rev0 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-adam-v1 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-atari-esp32-s3-wroom-1-n16r8 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-coco-esp32-s3-wroom-1-n8r8 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-coco-esp32-s3-wroom-1-n16r8 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-iec-fujiapple [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-iec-lolin-d32 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-iec-nugget [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-lynx-rev1 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujinet-rs232-s3 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujiversal-drivewire [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
sdkconfig.fujiversal-rs232 [sdkconfig] re-enable bounded TCP out-of-order queueing (#1615) (#1634) 2026-09-15 13:24:03 -05:00
upstream.sh fixup rebase for build.sh 2024-01-22 18:37:33 -05:00
verify-webui-progress.sh Consolidate webui configs with inheritance (#1621) 2026-09-09 14:50:59 -05:00
verify-webui.sh Consolidate webui configs with inheritance (#1621) 2026-09-09 14:50:59 -05:00
version_common.py Created common library for version getting scripts (#1442) 2026-07-11 15:51:15 -07:00

README.md

FujiNet

A multi-function peripheral built on ESP32 hardware being developed for the multiple 8-bit systems

Please see the GitHub wiki for documentation and additional details.

A dedicated web site is also available at https://fujinet.online/

MAJOR ANNOUNCEMENT FOR ANYONE WORKING ON FIRMWARE CODE:

Fujinet-platformio has been ported forward to the new PlatformIO Esp32 3.0. The changes made mean that it no longer builds under 1.12.x or 2.0, so you must upgrade in order to work on this firmware.

To upgrade:

  • Select Platforms from PIO Home in Quick Access

  • You should see an upgrade notice for Espressif 32. Upgrade it. After the upgrade, you will move from 1.12.x or 2.0 to 3.0.

  • Once this is done, delete your .vscode and .pio folders, and re-start vs.code.

Generating platformio configuration for your board

See build.sh documentation for full documentation on using build.sh to configure your platformio ini files.

ATARI

FujiNet currently provides the following devices for the Atari 8-bit system:

  • D: for disk emulation, allowing disk images to be read or written to on SD cards or TNFS servers over the local network or Internet.
  • P: for printing emulation, providing printer emulation for various types of popular printers, including Atari-branded, Epson, and other alternatives.
  • R: for RS-232 emulation, providing a Wi-Fi modem that can be used by existing communications programs that work with an Atari 850 interface.
  • N: providing a network adapter that can talk TCP, HTTP, UDP, and other protocols to other TCP/IP hosts.

This is the primary ESP32 firmware project. In addition, there are several related Github projects:

  • fujinet-config: Atari 8bit program to configure FujiNet
  • fujinet-nhandler: Atari 8bit "N:" device handler
  • fujinet-config-tools: Additional Atari 8bit programs to directly control FujiNet
  • fujinet-hardware: Schematics and design files for FujiNet hardware

Coleco ADAM

  • Disk and Tape emulation, allowing disk images to be read or written to on SD cards or TNFS servers over the local network or Internet.
  • Printing emulation, providing printer emulation for various types of popular printers, including Atari-branded, Epson, and other alternatives.
  • A network adapter that can talk TCP, HTTP, UDP, and other protocols to other TCP/IP hosts.

Apple

  • Floppy and Hard drive emulation
  • Printing emulation
  • An N device that allows Applesoft to use a network adapter that can talk TCP, HTTP, UDP, and other protocols to other TCP/IP hosts.

Other works in progress

Atari Lynx, IEC devices (Commodore 64, Commander X16), RC2014, S100 and more are coming all the time

How to contact us outside of GitHub:

There is active discussion and work on Discord: https://discord.gg/7MfFTvD

There are two active threads on the AtariAge forums: