Internal Search cannot search output in JupyterLab 4, is it an new feature?

I assume it is a feature in JupyterLab 4 to disable the search for stdout

Just curious, can we opt-out it if needed

Not sure what you mean? You can search output.

See Searching through Cell Outputs using CTRL+F in JupyterLab: Any Solutions? - #2 by afonit

(Here is what the ‘Find’ interface looks like in light form, from image in this post. See the funnel to the right of the cursor/hand in that picture. Toggle that.)