Skip to content

[Nexthop][Distro] Fix fboss_init.sh startup and package missing shared libs#1107

Open
benoit-nexthop wants to merge 1 commit intofacebook:mainfrom
nexthop-ai:fboss-init-fixes
Open

[Nexthop][Distro] Fix fboss_init.sh startup and package missing shared libs#1107
benoit-nexthop wants to merge 1 commit intofacebook:mainfrom
nexthop-ai:fboss-init-fixes

Conversation

@benoit-nexthop
Copy link
Copy Markdown
Contributor

Summary

fboss_init.sh wasn't sourcing setup_fboss_env, so weutil couldn't find its shared libraries and silently failed to generate fruid.json. Downstream services would then crash on startup because fruid.json was missing.

setup_fboss_env: derive FBOSS root from the script's own location instead of $PWD, and add an idempotency guard to prevent duplicate PATH and LD_LIBRARY_PATH prepends when the script is sourced multiple times.

fboss_init.sh: source setup_fboss_env at startup so weutil resolves its shared libraries; make generate_fruid() propagate failure with return 1; make main() exit 1 if fruid generation fails so systemd marks fboss_init.service as failed rather than succeeded.

fboss_cmd_find.sh: source setup_fboss_env before exec so every FBOSS command runs with the correct PATH and LD_LIBRARY_PATH.

package.py: the thrift-python migration (commit f9d8d65) switched FBOSS's manifest dependencies from the base variants (folly, wangle, fizz, mvfst, fmt, zstd, fbthrift) to their -python variants, which are built with BUILD_SHARED_LIBS=ON. As a result, every FBOSS binary (platform_manager, weutil, fsdb, ...) now links against libfolly.so, libwangle.so, libfmt.so, libfizz.so, libmvfst.so at runtime, not just Python extensions. Bundle these shared libraries in the forwarding-stack and platform-stack tarballs via a new _find_getdeps_libs() helper that locates .so files under each package's getdeps install prefix. LIB_NAME_OVERRIDES handles packages whose getdeps name differs from the library filename (e.g. fmt-python builds libfmt.so). Also add a lib64 -> lib symlink in each tarball so binaries with RPATH/RUNPATH pointing to lib64/ resolve correctly, and bundle libunwind.so from the LLVM build container since CentOS does not ship a compatible version and getdeps does not install it.

Test Plan

These fixes were needed to properly image DUTs with the latest distro image build. Without them no FBOSS agent would come up.

…package shared library dependencies

fboss_init.sh never sourced setup_fboss_env, so weutil couldn't find its
shared libraries and silently failed to generate fruid.json. Downstream
services (qsfp_service, sw agent) would then crash on startup because
fruid.json was missing.

setup_fboss_env: derive FBOSS root from the script's own location instead
of $PWD, and add an idempotency guard to prevent duplicate PATH and
LD_LIBRARY_PATH prepends when the script is sourced multiple times.

fboss_init.sh: source setup_fboss_env at startup so weutil resolves its
shared libraries; make generate_fruid() propagate failure with return 1;
make main() exit 1 if fruid generation fails so systemd marks
fboss_init.service as failed rather than succeeded.

fboss_cmd_find.sh: source setup_fboss_env before exec so every FBOSS
command runs with the correct PATH and LD_LIBRARY_PATH.

package.py: the thrift-python migration (commit f9d8d65, "CMake and
Docker changes for thrift-python migration") switched fboss's manifest
dependencies from the base variants (folly, wangle, fizz, mvfst, fmt,
zstd, fbthrift) to their -python variants, which are built with
BUILD_SHARED_LIBS=ON. As a result, every FBOSS binary (platform_manager,
weutil, fsdb, ...) now links against libfolly.so, libwangle.so, libfmt.so,
libfizz.so, libmvfst.so at runtime, not just Python extensions. Bundle
these shared libraries in the forwarding-stack and platform-stack tarballs
via a new _find_getdeps_libs() helper that locates .so files under each
package's getdeps install prefix. LIB_NAME_OVERRIDES handles packages
whose getdeps name differs from the library filename (e.g. fmt-python
builds libfmt.so). Also add a lib64 -> lib symlink in each tarball so
binaries with RPATH/RUNPATH pointing to lib64/ resolve correctly, and
bundle libunwind from the LLVM build container since CentOS does not ship
a compatible version and getdeps does not install it.
@benoit-nexthop benoit-nexthop requested a review from a team as a code owner April 18, 2026 20:30
@meta-cla meta-cla Bot added the CLA Signed label Apr 18, 2026
@benoit-nexthop
Copy link
Copy Markdown
Contributor Author

Chain of events that led us here:

  1. 518c49f (Jan 22) — Created -python manifest variants with BUILD_SHARED_LIBS=ON because Python extension wheels need -fPIC C++ libs. At this point, FBOSS was unaffected.
  2. 108845a (Mar 7) — Unconditionally enabled BUILD_SHARED_LIBS on gflags and glog (previously conditional on --shared_libs=on), and added -fPIC to libsodium, libunwind, xxhash, xz so they could be linked into shared libs. This made gflags/glog always shared, which is why libunwind is packaged in package.py.
  3. f9d8d65 (Apr 6) — At this point FBOSS binaries are going dynamic linked. To support the thrift-py to thrift-python migration (Cython extensions need folly-python and fbthrift-python for ABI compatibility), FBOSS's manifest was switched to the -python variants wholesale. Consequence: platform_manager, weutil, fsdb, etc. now all link against shared libs, therefore those shared libs now need to be bundled in the distro image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant