See this linked xterm.js github discussion. I think the solution outline is in the reply by jerch. Sorry for jumping to the conclusion that it would be an xterm.js bug.
JupyterHub Terminal gets clobbered with ascii color codes · xtermjs/xterm.js · Discussion #6155 · GitHub