AsyncFetcher Import Triggers Camoufox Dependency Error Despite Lightweight Installation

[simonszalai]

Have you searched if there an existing issue for this?

• I have searched the existing issues

Python version (python --version)

3.12

Scrapling version (scrapling.version)

0.3.5

Dependencies version (pip3 freeze)

aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
aiosqlite==0.21.0
alembic==1.16.4
annotated-types==0.7.0
anyio==4.9.0
apprise==1.9.3
asgi-lifespan==2.1.0
asyncpg==0.30.0
attrs==25.3.0
beautifulsoup4==4.13.5
browserforge==1.2.3
cachetools==6.1.0
camoufox==0.4.11
certifi==2025.7.14
cffi==1.17.1
charset-normalizer==3.4.2
click==8.3.0
cloudpickle==3.1.1
colorama==0.4.6
contourpy==1.3.3
coolname==2.2.0
cryptography==45.0.5
cssselect==1.3.0
curl-cffi==0.13.0
cycler==0.12.1
cython==3.1.4
dateparser==1.2.2
diskcache==5.6.3
distro==1.9.0
docker==7.1.0
docstring-parser==0.17.0
exceptiongroup==1.3.0
fastapi==0.116.1
filelock==3.19.1
fonttools==4.59.1
frozenlist==1.7.0
fsspec==2025.7.0
geoip2==5.1.0
graphql-core==3.2.6
graphviz==0.21
greenlet==3.2.3
griffe==1.8.0
grpcio==1.74.0
h11==0.16.0
h2==4.2.0
hpack==4.1.0
httpcore==1.0.9
httpx==0.28.1
httpx-sse==0.4.1
humanize==4.12.3
hyperframe==6.1.0
idna==3.10
importlib-metadata==8.7.0
instructor==1.11.3
jinja2==3.1.6
jinja2-humanize-extension==0.4.0
jiter==0.10.0
jsonpatch==1.33
jsonpointer==3.0.0
jsonschema==4.25.0
jsonschema-specifications==2025.4.1
kiwisolver==1.4.9
language-tags==1.2.0
lxml==6.0.2
mako==1.3.10
markdown==3.8.2
markdown-it-py==3.0.0
markupsafe==3.0.2
matplotlib==3.10.5
maxminddb==2.8.2
mcp==1.14.1
mdurl==0.1.2
msgspec==0.19.0
multidict==6.6.3
nodeenv==1.9.1
numpy==2.3.1
oauthlib==3.3.1
openai==1.98.0
opentelemetry-api==1.35.0
orjson==3.11.3
packaging==25.0
pandas==2.3.1
patchright==1.55.2
pathspec==0.12.1
pendulum==3.1.0
pglast==7.2
pgvector==0.4.1
pillow==11.3.0
platformdirs==4.4.0
playwright==1.55.0
postgres-mcp==0.3.0
prefect==3.4.14
prefect-github==0.3.1
prefect-sqlalchemy==0.5.3
prometheus-client==0.22.1
propcache==0.3.2
protobuf==6.31.1
psutil==7.0.0
psycopg==3.2.10
psycopg-binary==3.2.10
psycopg-pool==3.2.6
pycparser==2.22
pydantic==2.11.7
pydantic-core==2.33.2
pydantic-extra-types==2.10.5
pydantic-settings==2.10.1
pyee==13.0.0
pygments==2.19.2
pyobjc-core==11.1
pyobjc-framework-cocoa==11.1
pyparsing==3.2.3
pyright==1.1.403
pysocks==1.7.1
python-dateutil==2.9.0.post0
python-dotenv==1.1.1
python-multipart==0.0.20
python-slugify==8.0.4
python-socks==2.7.1
python-telegram-bot==22.3
pytz==2025.2
pyyaml==6.0.2
readchar==4.2.1
referencing==0.36.2
regex==2024.11.6
requests==2.32.4
requests-file==2.1.0
requests-oauthlib==2.0.0
rfc3339-validator==0.1.4
rich==14.0.0
rpds-py==0.26.0
ruamel-yaml==0.18.14
ruamel-yaml-clib==0.2.12
ruff==0.12.5
scrapling==0.3.5
screeninfo==0.8.1
semver==3.0.4
setuptools==80.9.0
sgqlc==17
shellingham==1.5.4
six==1.17.0
sniffio==1.3.1
soupsieve==2.8
sqlalchemy==2.0.41
sqlmodel==0.0.24
sse-starlette==3.0.2
starlette==0.47.2
tenacity==9.1.2
text-unidecode==1.3
tldextract==5.3.0
toml==0.10.2
tqdm==4.67.1
typer==0.16.0
typing-extensions==4.14.1
typing-inspection==0.4.1
tzdata==2025.2
tzlocal==5.3.1
ua-parser==1.0.1
ua-parser-builtins==0.18.0.post1
urllib3==2.5.0
uv==0.8.2
uvicorn==0.35.0
websockets==15.0.1
xai-sdk==1.0.1
yarl==1.20.1
zipp==3.23.0

What's your operating system?

macOS

Are you using a separate virtual environment?

Yes

Expected behavior

AsyncFetcher should work using httpx for HTTP requests without requiring Camoufox or other browser dependencies, as implied by the lightweight fetchers extra.

Actual behavior

The script fails with:

textFileNotFoundError: Version information not found at /root/.cache/camoufox/version.json. Please run camoufox fetch to install.

This is caused by unconditional imports of browser-related modules in fetchers.py:

pythonfrom scrapling.engines._browsers import (
    AsyncDynamicSession,
    DynamicSession,
    StealthySession,
    AsyncStealthySession,
)

Steps To Reproduce

To reproduce the issue with Scrapling's AsyncFetcher triggering a Camoufox dependency error, follow these steps:

1. Set Up a Clean Python Environment:

   • Create a new virtual environment to avoid conflicts:

     python -m venv venv
     source venv/bin/activate  # On Windows: venv\Scripts\activate
     

2. Install Scrapling with Fetchers:

   • Install Scrapling with the fetchers extra, which includes httpx but avoids browser dependencies:

     pip install "scrapling[fetchers]"
     

3. Create a Test Script:

   • Save the following code as test_fetcher.py:

     from scrapling.fetchers import AsyncFetcher
     import asyncio
     
     async def main():
         async with AsyncFetcher() as fetcher:
             page = await fetcher.get("https://example.com")
             print(page.html)
     
     asyncio.run(main())

4. Run the Script:

   • Execute the script:

     python test_fetcher.py
     

5. Observe the Error:

   • The script will fail with an error similar to:

     FileNotFoundError: Version information not found at /root/.cache/camoufox/version.json. Please run `camoufox fetch` to install.
     

   • This occurs because fetchers.py imports browser-related modules that check for Camoufox's version file, even though AsyncFetcher doesn't need it.

Environment Details:

• Python version: 3.12 (or compatible version)
• Scrapling version: Check with pip show scrapling (e.g., 0.2.0)
• OS: Any (e.g., Ubuntu, macOS, Windows, or Docker)
• No Camoufox or browser dependencies installed (i.e., camoufox fetch or scrapling install not run)

This reliably reproduces the issue, showing that AsyncFetcher cannot be used without Camoufox, despite being intended as a lightweight HTTP fetcher.

[D4Vinci]

Hi mate, thanks for this detailed report!

I see the issue is that camoufox is imported by mistake. I will look into it and resolve the problem with the upcoming update (within the next few days).

BTW, the AsyncFetcher uses curl_cffi starting from v0.3 instead of httpx, as before.

[D4Vinci]

Fixed with the current update