Skip to content

Troubleshooting

Common failures and how to diagnose them. Most issues fall into one of three buckets: build-time (nupkg not produced or wrong shape), publish-time (registry rejected the upload), or load-time (the WM can't load the nupkg after CreateWorker).

"build_managed.py fails with ERROR: .../Virtufin.Worker.WebSocketManagerController.nupkg not found. PackageWorker target may have failed."

The PackageWorker MSBuild target in the managed csproj didn't run. Check:

  1. Is the csproj path passed to build_managed.py correct? The script auto-discovers from the path you give it.
  2. Does dotnet publish itself succeed? The script prints the publish output; look for errors above the "ERROR" line.
  3. Does the <TargetFramework> in the csproj match the publish target? (Should be net10.0.)

"build_aot.py hangs on docker run"

Two common causes:

  1. The cross-build image isn't local. The script's maybe_pull_image only runs on --pull-image. Without that flag, the script will time out trying to pull. Run with --pull-image, or pre-pull: docker pull docker.haenerconsulting.com/library/dotnet-crossbuild:10.0.
  2. On Apple Silicon, the cross-build image runs via qemu-x86_64 emulation. This is slow (~15+ minutes for an AOT compile). The compile is happening, just slowly. Let it finish, or run on an x86_64 host.

"AOT publish fails with error MSB4184: ... GetTargetFrameworkVersion(net10.0, 2)"

The cross-build image's dotnet SDK doesn't have a TargetFramework resolver entry for the net10.0+RID combination. This is a known issue with the cross-build image on arm64 emulation. Workarounds:

  1. Use an x86_64 host (no qemu emulation, no MSB4184).
  2. Use the --docker-image '' flag to skip Docker and run dotnet publish directly on a host that has clang + cross-binutils installed natively.

"WM logs: Method not found: ... Virtufin.Worker.DevKit.IGatewayClient ... CommandWorker\1.get_Gateway()`"

The WM Docker image was built from an older DevKit version than the worker nupkg was built against. The new IGatewayClient interface and Gateway property are in the new DevKit; the old WM image doesn't know about them.

Fix: rebuild the WM Docker image (docker.haenerconsulting.com/virtufin/workmanager) against the new DevKit. The devkit-nuget.yaml workflow in the workmanager repo handles this; pushing to the workmanager repo's master triggers a re-publish of the DevKit NuGet AND a re-build of the WM image.

"binance.py raises UnsupportedWorkerRuntimeError: supports [linux-arm64, linux-x64], but you are on osx-arm64"

You're trying to load a Linux-only worker nupkg on macOS. The worker was built with linux-arm64,linux-x64 RIDs and the AOT path doesn't ship osx-* binaries.

Fix: 1. Build the worker with osx-arm64 as well: python3 scripts/build_aot.py --rids linux-arm64,linux-x64,osx-arm64. The script will reject osx-arm64 (no osx-* toolchain in the cross-build image) — so you actually need a Mac with the toolchain to build that RID. 2. OR run the worker in a Linux container. 3. OR use the managed path on macOS (which doesn't need the cross-build image at all): python3 scripts/build_managed.py. The managed DotNetDllEngine runs anywhere .NET 10 runs.

"binance.py raises Gateway invoke failed: ... 401"

The nupkg is now in a NuGet feed that requires authentication. The current binance.py and the WM's CodeFetcher HttpClient don't supply credentials.

Fix (out of scope for this repo — it's a registry configuration issue in virtufin-gitea): either make the nupkg public, or add Bearer auth to the WM's CodeFetcher HttpClient and the binance.py downloader.

"binance.py raises Gateway invoke failed: ... 404"

The URL pattern is wrong. Verify: - The nupkg version in build_version.txt matches a published version on the Gitea feed. - The script_url_template in config.json resolves to the right Gitea path (the package/... segment is NuGet v2 protocol; the download/... segment is v3).

"AOT smoke test fails: missing nupkg entries: [...]

The build script's PackageWorkerNative target didn't copy the .so into the staging dir. Check: - The AOT compile actually produced the .so (look at the dotnet publish output). - The AssemblyName in the AOT csproj matches the on-disk filename (it should, by default). - The build script's rename step (mv <AssemblyName>.so lib<AssemblyName>.so) ran. If the AOT output is named differently (e.g. <AssemblyName>Native.so), the rename no-ops and the nupkg has the wrong filename.

"AOT smoke test fails: 404 NOT exported (got: [...])"

The .so was produced but the [UnmanagedCallersOnly] exports aren't there. Check the AOT shim's Process and FreeResult methods have the correct attributes:

[UnmanagedCallersOnly(EntryPoint = "Process")]
public static unsafe int Process(
    IntPtr host, IntPtr inBuf, int inLen, IntPtr outBuf, IntPtr outLen)
    => AotNative<MyWorkerNative>.ProcessStatic(
        host, inBuf, inLen, (IntPtr*)outBuf, (int*)outLen);

[UnmanagedCallersOnly(EntryPoint = "FreeResult")]
public static void FreeResult(IntPtr result)
    => AotNative<MyWorkerNative>.FreeResultStatic(result);

If the methods aren't there or the entry-point names differ, the AOT compiler may have stripped them as unused. The NativeAOT analyzer (<EnableAotAnalyzer>true</EnableAotAnalyzer>) flags these at build time.

"Docker pull hangs on the cross-build image"

The registry is reachable (HEAD works) but the layer download stalls. This is a known intermittent issue with the local registry mirror. Mitigations: 1. Retry with a delay. 2. Run on a host with a more direct network path to the registry. 3. Pre-pull the image and pass --docker-image '' to the build script to use the host toolchain instead (requires clang and aarch64-linux-gnu-objcopy / x86_64-linux-gnu-objcopy on $PATH).

"binance.py hangs after the first worker loads"

The WM's CodeFetcher HttpClient is trying to fetch the nupkg but the request is timing out. Check: - The URL is reachable from the WM container (DNS, network). - The nupkg version exists in the feed. - The HttpClient timeout (CodeFetchTimeoutSeconds in virtufin-workmanager) hasn't been hit.

The WM log will show the HTTP failure (HttpRequestException or similar) at the time the worker is loaded.

See also