Advanced Docker Buildx Debugging Guide: Using –on-error
Introduction to Buildx –on-error
The --on-error
flag in Docker Buildx is a powerful debugging feature that allows you to inspect the build container’s state when a build step fails. This is particularly useful for:
- Investigating build failures in complex Dockerfiles
- Debugging package installation issues
- Troubleshooting configuration problems
- Examining file system state at the point of failure
Basic Usage
1. Simple Example with –on-error
1
2
3
4
| FROM ubuntu:22.04
RUN apt-get update
RUN apt-get install -y nonexistent-package
RUN echo "This won't run due to previous error"
|
Debug command:
1
| docker buildx build --progress=plain --on-error=continue .
|
When the build fails, you’ll get a shell in the container:
1
2
3
4
5
| #18 [3/4] RUN apt-get install -y nonexistent-package
#18 ERROR: process "apt-get install -y nonexistent-package" did not complete successfully: exit code: 100
>>> Entering debug shell ...
root@f8d9a2b3c4:/#
|
2. Interactive Debugging
Once in the debug shell, you can:
1
2
3
4
5
6
7
8
9
10
11
| # Check package availability
root@f8d9a2b3c4:/# apt-cache search your-package
# View logs
root@f8d9a2b3c4:/# cat /var/log/apt/term.log
# Test commands manually
root@f8d9a2b3c4:/# apt-get install -y your-package
# Check environment variables
root@f8d9a2b3c4:/# env
|
Advanced Usage Examples
1. Debugging Complex Build Dependencies
1
2
3
4
5
6
| FROM node:16
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build # Assuming this fails
|
Debug command with additional options:
1
2
3
4
5
| docker buildx build \
--progress=plain \
--on-error=continue \
--no-cache \
--build-arg NODE_ENV=development .
|
2. Debugging Multi-stage Builds
1
2
3
4
5
6
7
8
9
| FROM golang:1.20 AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN go build -v # If this fails
FROM alpine:3.18
COPY --from=builder /app/myapp /usr/local/bin/
|
Debug command for specific stage:
1
2
3
4
| docker buildx build \
--progress=plain \
--on-error=continue \
--target builder .
|
3. Debugging with BuildKit Cache
1
2
3
4
5
6
7
| FROM python:3.9
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
COPY . .
RUN python setup.py build # Debug this step
|
Debug command with cache options:
1
2
3
4
5
| docker buildx build \
--progress=plain \
--on-error=continue \
--cache-from type=local,src=/tmp/cache \
--cache-to type=local,dest=/tmp/cache .
|
Best Practices for –on-error Debugging
1. Using –progress Options
1
2
3
4
5
6
7
8
| # Default progress output
docker buildx build --on-error=continue .
# Plain text output (most detailed)
docker buildx build --progress=plain --on-error=continue .
# Detailed output with timing
docker buildx build --progress=plain --on-error=continue --no-cache .
|
2. Combining with Other Debug Flags
1
2
3
4
5
6
7
| # Maximum verbosity
docker buildx build \
--progress=plain \
--on-error=continue \
--no-cache \
--build-arg BUILDKIT_STEP_LOG_MAX_SIZE=-1 \
--build-arg BUILDKIT_STEP_LOG_MAX_SPEED=-1 .
|
3. Debug Environment Setup
1
2
3
4
5
6
7
8
9
| # Add debugging tools in your Dockerfile
RUN apt-get update && apt-get install -y \
vim \
curl \
netcat \
strace \
tcpdump
# Use these in your debug session
|
Common Debugging Scenarios
1. Network Issues
1
2
3
4
| # In debug shell
root@f8d9a2b3c4:/# ping -c 3 google.com
root@f8d9a2b3c4:/# curl -v https://registry.npmjs.org
root@f8d9a2b3c4:/# netstat -tulpn
|
2. File System Issues
1
2
3
4
| # In debug shell
root@f8d9a2b3c4:/# ls -la
root@f8d9a2b3c4:/# df -h
root@f8d9a2b3c4:/# find / -name "missing-file"
|
3. Permission Issues
1
2
3
4
| # In debug shell
root@f8d9a2b3c4:/# id
root@f8d9a2b3c4:/# ls -l /problem/directory
root@f8d9a2b3c4:/# namei -l /path/to/file
|
Exiting and Cleanup
1
2
3
4
5
| # Exit the debug shell
root@f8d9a2b3c4:/# exit
# Clean up afterward
docker buildx prune
|
Tips and Tricks
- Preserve Debug Container
1
2
| # Start new build with preserved debug container
docker buildx build --progress=plain --on-error=continue --keep .
|
- Export Debug Container
1
2
3
| # In another terminal while in debug session
docker commit <container-id> debug-image
docker run -it debug-image sh
|
- Create Debug Snapshot
1
2
3
4
| # In debug shell
tar czf /tmp/debug-snapshot.tar.gz /var/log /etc
# In another terminal
docker cp <container-id>:/tmp/debug-snapshot.tar.gz .
|
Limitations and Considerations
- Resource Usage
- Debug sessions keep containers running
- Monitor disk space and memory usage
- Clean up debug containers regularly
- Security
- Don’t leave debug tools in production images
- Be cautious with sensitive information in debug sessions
- Remove debug artifacts before pushing images
- Performance
- Debug builds are slower due to additional logging
- Cache invalidation might occur more frequently
- Consider using –cache-from for faster rebuilds