Known Issues on the HTC
This page documents some common and known issues encountered on the HTC system. While this page can be beneficial in troubleshooting, it does not contain a comprehensive list of errors.
General
When submitting a job, it doesn't run / goes on hold and shows the error "Job credentials are not available".
Cause:
This is a complicated bug that can strike randomly. We're working on a fix.
Solution:
To work around this issue, run the following command on the access point before resubmitting the job.
echo | condor_store_cred add-oauth -s scitokens -i -
My job exits with an "Illegal instructions" error.
Cause:
The instructions for communicating with the processor does not match between software and hardware.
Solution:
In the submit file, try adding one of the following to your requirements line:
requirements = (has_avx || has_avx2)
requirements = has_avx2
requirements = (Microarch > x86-64-v3)
Then resubmit and check if this resolves the issue.
Containers
When building an Apptainer, "apt" commands in the %post block fail to run.
Example error message:
Couldn't create temporary file /tmp/apt.conf.9vQdLs for passing config to apt-key
Cause:
The container needs global read/write permissions in order to update or install packages using the apt command.
Solution:
Add chmod 777 /tmp to the front of your %post block. See the example below:
Bootstrap: docker
From: ubuntu:22.04
%post
chmod 777 /tmp
apt-get update -y
We also recommend using the -y option to prevent installation from hanging due to interactive prompts.
After submitting a job with a Docker container, it goes on hold with a message like, "Cannot pull image user/repo:tag".
Cause:
The Docker container is likely built on an Apple computer using an ARM processor, which is incompatible with Linux machines.
Solution:
To resolve this, when building your Docker container, use the command:
docker build --platform linux/amd64 .
My interactive Apptainer job is failing with the error message, "Can't open master pty Bad file descriptor".
Cause:
Interactive Apptainer jobs are incompatible with CentOS7 machines.
Solution:
To your submit file, add the following line and resubmit:
requirements = (OpSysMajorVer > 7)
Can't find your issue?
Visit our Get Help page.