Debugging CGo Parse Error in Go Build Process - macOS ARM64

Problem Description

When attempting to build a Go project (using gopy ) with dynamically linked libraries using the following command:

go build -mod=mod -buildmode=c-shared -ldflags=-s -w -extldflags='-lresolv' -o orcalogparser_go.so .

The following error was encountered:

cgo: cannot parse $WORK/b001/_cgo_.o as ELF, Mach-O, PE or XCOFF

Troubleshooting Steps

1. Verified Go Version : Ensured that the Go version in the CI environment matched the local development environment.

2. Checked for Cross-Compilation Issues : Investigated potential discrepancies between the build environment and the target platform.

3. Investigate Gopy : Forked gopy project and enabled verbose logging and flag changes to see if it compiles.

4. Reviewed Environment Variables : Examined GOOS and GOARCH settings to ensure they matched the target platform.

5. Review xcode : Examined if xcode-select (xcode tools) were installed properly and path was set. Verified if clang is working as expected

6. Verified Dependencies : Confirmed that all required dependencies, including the resolv library, were available in the CI environment.

   
   

   Apparently starting Go1.20 - the macOS link of go c-archive requires lresolv to be specified. More info here

   
   

   It’s also confusing where exactly to specify this ( ref ) - as it needs to be specified inside external flags -extldflags which should come under -ldflags

   
   

   Example usage:

   go build -trimpath -buildmode=c-archive -ldflags '-w -s -extldflags "-lresolv"'
   

   
   

7. Attempted Clean Build : Tried cleaning the build cache and rebuilding:

   go clean -cache -modcache -i -r
   

   
   

8. Used Verbose Output : Ran the build command with the -x flag for more detailed output:

   go build -x -mod=mod -buildmode=c-shared -ldflags="-s -w" -extldflags="-lresolv" -o orcalogparser_go.so .
   

   
   

9. Check python sysconfig : Python header files when installed on a system have flags set to determine the build architecture and compilations like shflags, shlinks.

   
   

   Example from my local system:

   {"version": 3, "minor": 11, "incdir": "/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11", "libdir": "/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/lib", "libpy": "libpython3.11.a", "shlibs": "-ldl  -framework CoreFoundation", "syslibs": "", "shlinks": "-Wl,-stack_size,1000000  -framework CoreFoundation /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python", "shflags": " -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk", "extsuffix": ".cpython-311-darwin.so"}
   

   
   

   To log the values like above, use:

   echo "$(python -c "import sysconfig; print('\n'.join([f'{k}={v}' for k,v in sysconfig.get_config_vars().items()]))")"
   

   
   

10. Error with invalid flag in CGo

    invalid flag in #cgo LDFLAGS: -Wl,--rpath=/opt/hostedtoolcache/Python/3.11.10/x64/lib
    

    
    

    Was resolved using CGO_LDFLAGS_ALLOW flag which takes in a regex. Apparently from go1.10 ( ref ) - a safelist of linker/compile options were introduced during go get, build, etc.

    
    

    Thus when CGo wants to pass any other option - we need to specify a regex to whitelist it.

    
    

    Example usage:

    CGO_LDFLAGS_ALLOW='\-undefined|dynamic_lookup|\-extldflags=.*'
    

    
    

11. Investigated CI Environment Limitations : Researched potential restrictions in the CI environment that might affect the build process.

12. Enabling verbose build logs - We enabled verbose build logs (go build -x), which revealed the following crucial information:

    
    

    TERM='dumb' clang -I . -fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=$WORK/b001=/tmp/go-build -gno-record-gcc-switches -fno-common -o $WORK/b001/_cgo_.o $WORK/b001/_cgo_main.o $WORK/b001/_x001.o $WORK/b001/_x002.o -O2 -g -extldflags=-lresolv -undefined dynamic_lookup -arch arm64 -arch x86_64 -g
    

    
    

    This showed conflicting architecture flags (-arch arm64 and -arch x86_64) being set.

Solution 🚀

The issue was resolved by setting the ARCHFLAGS environment variable:

export ARCHFLAGS="-arch arm64"

This explicitly specified the target architecture for the build process, ensuring consistency with the local development environment (macOS ARM64).

Previously, it was attempting universal build with both x84_64 and arm64 set under LDSHARED flag.

This caused the build to fail - as the binaries were built only for arm64 and using buildmode=c-shared

Recommended Build Configuration

For future builds, especially in CI environments, consider using the following configuration to ensure consistency:

export ARCHFLAGS="-arch arm64" export GOOS=darwin export GOARCH=arm64 go build -x -mod=mod -buildmode=c-shared -ldflags="-s -w" -o orcalogparser_go.so .

And if using buildmode=c-archive use:

export ARCHFLAGS="-arch arm64" export GOOS=darwin export GOARCH=arm64 go build -x -mod=mod -buildmode=c-shared -ldflags="-s -w" -extldflags="-lresolv" -o orcalogparser_go.so .

When using Github Actions use:

name: Generate Python bindings  
  env:  
    CGO_ENABLED: 1  
    GOOS: ${{ matrix.runner.os }}  
    GOARCH: ${{ matrix.runner.arch }}  
    ARCHFLAGS: "-arch ${{ matrix.runner.arch }}"  
    CGO_LDFLAGS_ALLOW: '.*'
  run: >  
    gopy pkg ... -dynamic-link=true -symbols=false
    github.com/orcasecurity/my-project/pkg/hello

Note:

I've used a relaxed regex for ldflags here - ".*"

You can make it more stricter, something like - "\-|dynamic_lookup|-Wl|--rpath=.*"