Running Playwright in JupyterLab Notebook Problem - Not implemented Error

Stuck on trying to get Playwright to work in Jupyter labs on Windows … Jupyter Labs 3.5.2, Python 3.9.15, Playwright 1.29.0. Code is …

import asyncio

from playwright.async_api import async_playwright

async def main():
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        await page.goto("http://playwright.dev")
        print(await page.title())
        await browser.close()
        
await main()

Error is:

Task exception was never retrieved
future: <Task finished name='Task-5' coro=<Connection.run() done, defined at C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_connection.py:240> exception=NotImplementedError()>
Traceback (most recent call last):
  File "C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_connection.py", line 247, in run
    await self._transport.connect()
  File "C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_transport.py", line 132, in connect
    raise exc
  File "C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_transport.py", line 120, in connect
    self._proc = await asyncio.create_subprocess_exec(
  File "C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\asyncio\subprocess.py", line 236, in create_subprocess_exec
    transport, protocol = await loop.subprocess_exec(
  File "C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\asyncio\base_events.py", line 1676, in subprocess_exec
    transport = await self._make_subprocess_transport(
  File "C:\Users\Larry\anaconda3\envs\TDA_virtual\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[1], line 16
     13         print(await page.title())
     14         await browser.close()
---> 16 await main()

Cell In[1], line 9, in main()
      8 async def main():
----> 9     async with async_playwright() as p:
     10         browser = await p.chromium.launch()
     11         page = await browser.new_page()

File ~\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\async_api\_context_manager.py:46, in PlaywrightContextManager.__aenter__(self)
     44 if not playwright_future.done():
     45     playwright_future.cancel()
---> 46 playwright = AsyncPlaywright(next(iter(done)).result())
     47 playwright.stop = self.__aexit__  # type: ignore
     48 return playwright

File ~\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_connection.py:247, in Connection.run(self)
    244 async def init() -> None:
    245     self.playwright_future.set_result(await self._root_object.initialize())
--> 247 await self._transport.connect()
    248 self._init_task = self._loop.create_task(init())
    249 await self._transport.run()

File ~\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_transport.py:132, in PipeTransport.connect(self)
    130 except Exception as exc:
    131     self.on_error_future.set_exception(exc)
--> 132     raise exc
    134 self._output = self._proc.stdin

File ~\anaconda3\envs\TDA_virtual\lib\site-packages\playwright\_impl\_transport.py:120, in PipeTransport.connect(self)
    117     if getattr(sys, "frozen", False):
    118         env.setdefault("PLAYWRIGHT_BROWSERS_PATH", "0")
--> 120     self._proc = await asyncio.create_subprocess_exec(
    121         str(self._driver_executable),
    122         "run-driver",
    123         stdin=asyncio.subprocess.PIPE,
    124         stdout=asyncio.subprocess.PIPE,
    125         stderr=_get_stderr_fileno(),
    126         limit=32768,
    127         creationflags=creationflags,
    128         env=env,
    129     )
    130 except Exception as exc:
    131     self.on_error_future.set_exception(exc)

File ~\anaconda3\envs\TDA_virtual\lib\asyncio\subprocess.py:236, in create_subprocess_exec(program, stdin, stdout, stderr, loop, limit, *args, **kwds)
    229     warnings.warn("The loop argument is deprecated since Python 3.8 "
    230                   "and scheduled for removal in Python 3.10.",
    231                   DeprecationWarning,
    232                   stacklevel=2
    233     )
    234 protocol_factory = lambda: SubprocessStreamProtocol(limit=limit,
    235                                                     loop=loop)
--> 236 transport, protocol = await loop.subprocess_exec(
    237     protocol_factory,
    238     program, *args,
    239     stdin=stdin, stdout=stdout,
    240     stderr=stderr, **kwds)
    241 return Process(transport, protocol, loop)

File ~\anaconda3\envs\TDA_virtual\lib\asyncio\base_events.py:1676, in BaseEventLoop.subprocess_exec(self, protocol_factory, program, stdin, stdout, stderr, universal_newlines, shell, bufsize, encoding, errors, text, *args, **kwargs)
   1674     debug_log = f'execute program {program!r}'
   1675     self._log_subprocess(debug_log, stdin, stdout, stderr)
-> 1676 transport = await self._make_subprocess_transport(
   1677     protocol, popen_args, False, stdin, stdout, stderr,
   1678     bufsize, **kwargs)
   1679 if self._debug and debug_log is not None:
   1680     logger.info('%s: %r', debug_log, transport)

File ~\anaconda3\envs\TDA_virtual\lib\asyncio\base_events.py:498, in BaseEventLoop._make_subprocess_transport(self, protocol, args, shell, stdin, stdout, stderr, bufsize, extra, **kwargs)
    494 async def _make_subprocess_transport(self, protocol, args, shell,
    495                                      stdin, stdout, stderr, bufsize,
    496                                      extra=None, **kwargs):
    497     """Create subprocess transport."""
--> 498     raise NotImplementedError

NotImplementedError: 

Have tried everything I have found including modifying init.py of Asyncio with each of the commented line of the following:

import asyncio

if sys.platform == 'win32':
    from .windows_events import *
#    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
#    asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
    __all__ += windows_events.__all__
else:
    from .unix_events import *  # pragma: no cover
    __all__ += unix_events.__all__

So far no joy … anyone help would be appreciated.

I believe the error comes from the following snippet from Playwright _transport.py routine

 try:
            # For pyinstaller
            env = get_driver_env()
            if getattr(sys, "frozen", False):
                env.setdefault("PLAYWRIGHT_BROWSERS_PATH", "0")

            self._proc = await asyncio.create_subprocess_exec(
                str(self._driver_executable),
                "run-driver",
                stdin=asyncio.subprocess.PIPE,
                stdout=asyncio.subprocess.PIPE,
                stderr=_get_stderr_fileno(),
                limit=32768,
                creationflags=creationflags,
                env=env,
            )
        except Exception as exc:
            self.on_error_future.set_exception(exc)
            raise exc

and I am thinking that the IOLoop from JupyterLabs is selectoreventlooppolicy and does not support pipes, perhaps that is what the “Not Implemented” is about, but I am still digging into this and into if it this is true and if the behaviour can be modified.

Again, all input is appreciated.

Cross-posted yesterday here.

0

It appears that the problem is specific to Windows Platform and as identified in the original question as the use of SelectorEventLoopPolicy rather than ProactorEventloopPolicy in JupyterLabs is the source of the issue.

Since SelectorEventLoopPolicy does not support Pipes the Not implemented error occurs when Playwright first tries to open a pipe. This link identifies that the EventLoopPolicy for JupyterLab is set in the routine kernelapp.py of ipykernel.

        if sys.platform.startswith("win") and sys.version_info >= (3, 8):
            import asyncio

            try:
                from asyncio import (
                    WindowsProactorEventLoopPolicy,
                    WindowsSelectorEventLoopPolicy,
                )
            except ImportError:
                pass
                # not affected
            else:
                if type(asyncio.get_event_loop_policy()) is WindowsProactorEventLoopPolicy:
                    # WindowsProactorEventLoopPolicy is not compatible with tornado 6
                    # fallback to the pre-3.8 default of Selector
#Comment out  - 637              asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())
#Add this to complete the if     print(f'do not change to Selector Event Loop')

Not sure if this will have any adverse effects on other operations, but it does cause the original code to execute in JupyterLab.

I implemented this as separate environment in case it has side effects.

1 Like