-
Notifications
You must be signed in to change notification settings - Fork 52
[CI] Refactor CICD test workflows #1862
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 173 commits
Commits
Show all changes
208 commits
Select commit
Hold shift + click to select a range
d3f32fa
modify build
mengfei25 bdc58d7
modify ut
mengfei25 b07b490
modify e2e
mengfei25 7b4582b
update
mengfei25 3ae4b09
update
mengfei25 fe06ca3
update
mengfei25 be531f7
Update nightly_ondemand.yml
mengfei25 1df6138
update
mengfei25 9fe4dcb
update
mengfei25 ef91984
update
mengfei25 01fbe46
update
mengfei25 191b5c0
update
mengfei25 f313b85
update
mengfei25 7d4488b
update
mengfei25 66e28da
update
mengfei25 8b22418
update
mengfei25 acf94d1
update
mengfei25 a6fa7da
update
mengfei25 053bed3
update
mengfei25 7ee8d4b
update
mengfei25 428e483
update
mengfei25 c483968
update
mengfei25 32474f8
update
mengfei25 5976099
update
mengfei25 28e53b2
update
mengfei25 2e9921e
update
mengfei25 b058b1a
update
mengfei25 93e5444
update
mengfei25 2aa5b11
Merge branch 'main' into mengfeil/containerd
mengfei25 8baec84
update
mengfei25 95709b9
Merge branch 'mengfeil/containerd' of https://github.com/intel/torch-…
mengfei25 d4c78aa
update
mengfei25 a8154f1
update
mengfei25 f25ecfe
update
mengfei25 e06e1bd
update
mengfei25 c437f29
update
mengfei25 0ae0bb1
update
mengfei25 d4da95d
update
mengfei25 db17d7d
update
mengfei25 b9c247a
update
mengfei25 a7d76ae
Merge branch 'main' into mengfeil/containerd
mengfei25 9ae98ea
update
mengfei25 c06f1ee
update
mengfei25 6e14f8b
update
mengfei25 9a621c5
update
mengfei25 bb17bab
update
mengfei25 981c744
update
mengfei25 6482077
update
mengfei25 ec0c1f2
update
mengfei25 1cc986e
update
mengfei25 b2b48c5
update
mengfei25 bccec93
update
mengfei25 9f604a7
update
mengfei25 8a78c7c
update
mengfei25 46d00c8
update
mengfei25 e3949d8
update
mengfei25 3f69213
update
mengfei25 e8b015a
update
mengfei25 bbd82cd
update
mengfei25 c144bab
get runner
mengfei25 40180c0
test env
mengfei25 54ea2f0
update
mengfei25 9b660b9
Revert "update"
mengfei25 7d025c0
update
mengfei25 dd23ceb
update
mengfei25 f21e4c9
Merge branch 'main' into mengfeil/containerd
mengfei25 de4a432
update
mengfei25 517c324
Merge branch 'mengfeil/containerd' of https://github.com/intel/torch-…
mengfei25 65cc01a
remove useless inputs for op benchmark
mengfei25 f727ef8
checkout torch-xpu-ops
mengfei25 18ada97
modify get runner
mengfei25 018f968
modify build
mengfei25 93fa112
modify build
mengfei25 ad8cc67
update
mengfei25 de0f557
Merge branch 'main' into mengfeil/containerd
mengfei25 7c9d3a3
update
mengfei25 2fc3b8e
update
mengfei25 78cedbf
update
mengfei25 c6bc928
update
mengfei25 9765fac
update
mengfei25 eda9634
modify ut
mengfei25 ec697f5
modify build
mengfei25 c1e4ca7
modify build
mengfei25 50e40fe
modify build
mengfei25 6173798
Merge branch 'main' into mengfeil/containerd
mengfei25 b3f6f0e
update
mengfei25 5848944
Merge branch 'mengfeil/containerd' of https://github.com/intel/torch-…
mengfei25 42da693
modify build
mengfei25 26b56db
modify build
mengfei25 77d8172
modify build
mengfei25 e9d551a
update
mengfei25 9649dfd
update
mengfei25 84a5132
update
mengfei25 ee18a1c
update
mengfei25 43fee42
update
mengfei25 e8f1c0d
update
mengfei25 517b081
update
mengfei25 ddecdf9
update
mengfei25 d1bf4cf
modify ut
mengfei25 d99668c
update
mengfei25 d0d1ceb
update
mengfei25 1f26538
update
mengfei25 d06b8db
update
mengfei25 70577e1
update
mengfei25 2467e9e
update
mengfei25 96ff039
update
mengfei25 da12ea0
update
mengfei25 4f6ecfd
modify ut
mengfei25 ba97507
modify ut
mengfei25 9896441
fix pip warnings
mengfei25 50467ee
modify ut logs path
mengfei25 5c62bc9
modify ut logs path
mengfei25 8b33c21
set run name for nightly and on-demand tests
mengfei25 f08c528
modify ut logs path
mengfei25 55bd5dc
ut summray always
mengfei25 dbd3a27
fix ut logs path
mengfei25 2e7680d
fix e2e summary permission
mengfei25 0a78df1
fix ut log path
mengfei25 074992f
update
mengfei25 b11510f
update
mengfei25 a18995b
modify e2e summary
mengfei25 754202d
modify e2e summary
mengfei25 27c5cff
modify e2e summary
mengfei25 92d7ff1
update
mengfei25 0fade31
Merge branch 'main' into mengfeil/containerd
mengfei25 56520ca
update
mengfei25 9117a0c
update
mengfei25 79155e5
Merge branch 'main' into mengfeil/containerd
mengfei25 587aa95
update
mengfei25 3b0b94d
update
mengfei25 e47b3e4
update
mengfei25 eafefa4
Merge branch 'main' into mengfeil/containerd
mengfei25 75c99ff
update
mengfei25 e244cb1
Update pull.yml
mengfei25 47cbdf5
modify pt2e
mengfei25 37e652e
Merge branch 'main' into mengfeil/containerd
mengfei25 de15a3f
update
mengfei25 8445b8b
e2e test matrix tests
mengfei25 a145fa2
Merge branch 'main' into mengfeil/containerd
mengfei25 8fe34c5
modify e2e summary
mengfei25 bfc98da
update
mengfei25 8c66acd
Merge branch 'main' into mengfeil/containerd
mengfei25 8064126
update
mengfei25 1ea6a62
update
mengfei25 530af25
update
mengfei25 091678f
update
mengfei25 eaa4bc4
update
mengfei25 f70ef8a
update deps
mengfei25 a12045a
update
mengfei25 70415c2
modify cache dir
mengfei25 5fcc6c6
update
mengfei25 263a393
Merge branch 'main' into mengfeil/containerd
mengfei25 18f22e0
update
mengfei25 2eb33ed
Merge branch 'mengfeil/containerd' of https://github.com/intel/torch-…
mengfei25 0add64e
update
mengfei25 8902540
update
mengfei25 0eda9f7
update
mengfei25 0aab07a
update
mengfei25 29a9fd8
update
mengfei25 c69854f
update
mengfei25 3d98b0e
Merge branch 'main' into mengfeil/containerd
mengfei25 a6b2302
merge main
mengfei25 0a17050
update
mengfei25 a7257b0
modify e2e summary
mengfei25 88af21f
Merge branch 'main' into mengfeil/containerd
mengfei25 8a54cfa
modify on-demand test
mengfei25 82783de
Merge branch 'mengfeil/containerd' of https://github.com/intel/torch-…
mengfei25 23f097f
modify on-demand test
mengfei25 a047acc
rebase
mengfei25 21aedbf
Merge branch 'main' into mengfeil/containerd
mengfei25 7df6ea3
rebase
mengfei25 946e547
Merge branch 'main' into mengfeil/containerd
mengfei25 0605da8
update
mengfei25 da64dea
remove env_prepare
mengfei25 0fa9e2b
remove duplicate
mengfei25 303883b
modify torch-xpu-ops parameter desc
mengfei25 1e0bcd4
rename get-runner
mengfei25 eb5f563
build torchvision and torchaudio with GCC11
mengfei25 c7ccbc7
remove rm -rf workspace if checkout
mengfei25 8639697
modify prepare env
mengfei25 695a9f1
use ubuntu-24.04 instead of latest
mengfei25 247adc6
Merge branch 'main' into mengfeil/containerd
mengfei25 a058b97
modify deps installation
mengfei25 6498c26
remove specific oneapi
mengfei25 2dc5f37
modify test_type parameter
mengfei25 a741aba
modify ut name
mengfei25 3fab342
update
mengfei25 00f823c
modify ut
mengfei25 c08b77a
modify ut
mengfei25 920baaf
modify
mengfei25 301dc74
modify
mengfei25 63ff56f
update
mengfei25 441914c
update
mengfei25 c8b76cf
Merge branch 'main' into mengfeil/containerd
mengfei25 64b2934
modify triton build
mengfei25 b5e11df
unify timeout
mengfei25 9120379
Merge branch 'main' into mengfeil/containerd
mengfei25 769fcb1
update
mengfei25 adfadb0
Merge branch 'mengfeil/containerd' of https://github.com/intel/torch-…
c17d557
lint
mengfei25 1a1b954
update
mengfei25 5550885
update
mengfei25 c0eb86d
update
mengfei25 4b553b0
update
mengfei25 4cb5fb5
modify torchao
mengfei25 1b3be5c
skip distributed case failed on rolling
zxd1997066 2b915b9
rebase
mengfei25 d39164b
Merge branch 'main' into mengfeil/containerd
mengfei25 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
name: Get Runner Infos | ||
|
||
outputs: | ||
runner_id: | ||
value: ${{ steps.runner.outputs.runner_id }} | ||
user_id: | ||
value: ${{ steps.runner.outputs.user_id }} | ||
render_id: | ||
value: ${{ steps.runner.outputs.render_id }} | ||
hostname: | ||
value: ${{ steps.runner.outputs.hostname }} | ||
|
||
permissions: read-all | ||
|
||
runs: | ||
using: composite | ||
steps: | ||
- name: Get runner | ||
shell: bash -xe {0} | ||
id: runner | ||
run: | | ||
# get test runner | ||
echo "runner_id=$(echo ${RUNNER_NAME} |sed 's/\-[0-9]$//')" |tee -a ${GITHUB_OUTPUT} | ||
echo "user_id=$(id -u)" |tee -a ${GITHUB_OUTPUT} | ||
echo "render_id=$(getent group render |cut -d: -f3)" |tee -a ${GITHUB_OUTPUT} | ||
echo "hostname=$(hostname)" |tee -a ${GITHUB_OUTPUT} | ||
# show host info | ||
lscpu | ||
lshw -C display | ||
free -h | ||
df -h | ||
cat /etc/os-release | ||
uname -a | ||
- name: Cleanup host | ||
shell: bash -xe {0} | ||
run: | | ||
# clean docker cache | ||
docker system prune -af || true | ||
# clean workspace | ||
ls -al | ||
sudo find ./ |grep -v "^\./$" |xargs sudo rm -rf | ||
cd ${RUNNER_WORKSPACE}/.. | ||
if [ "${PWD}" != "/" ];then | ||
ls -al | ||
sudo chmod 777 -R torch-xpu-ops _temp _actions _tool || true | ||
sudo rm -rf _temp | ||
fi |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
name: Linux E2E Test | ||
|
||
inputs: | ||
env_prepare: | ||
required: false | ||
description: If set to any value, will prepare suite test env | ||
suite: | ||
required: true | ||
type: string | ||
default: 'huggingface' | ||
description: Dynamo benchmarks test suite. huggingface,timm_models,torchbench. Delimiter is comma | ||
dt: | ||
required: true | ||
type: string | ||
default: 'float32' | ||
description: Data precision of the test.float32,bfloat16,float16,amp_bf16,amp_fp16. Delimiter is comma | ||
mode: | ||
required: true | ||
type: string | ||
default: 'inference' | ||
description: inference,training. Delimiter is comma | ||
scenario: | ||
required: true | ||
type: string | ||
default: 'accuracy' | ||
description: accuracy,performance. Delimiter is comma | ||
|
||
runs: | ||
using: composite | ||
steps: | ||
- name: E2E Test (${{ inputs.suite }} ${{ inputs.dt }} ${{ inputs.mode }} ${{ inputs.scenario }}) | ||
shell: bash -x {0} | ||
run: | | ||
pip list |grep -E 'intel|torch' | ||
cp ./.github/scripts/inductor_xpu_test.sh ./pytorch | ||
cd ./pytorch | ||
# check param | ||
function contains() { | ||
contains_status="echo 'Start $2 ...'" | ||
{ | ||
[[ $1 =~ (^|,)$2($|,) ]] | ||
} || { | ||
echo "[Warning] $2 is not suppotted type! Skipped!" | ||
contains_status="continue" | ||
} | ||
} | ||
xpu_num=$(clinfo --list |awk 'BEGIN{gpu=0;}{if(gpu==1 && $0~/Platform/){gpu=0;}; if(gpu==1){print $0;}; if($0~/Platform.*Graphics/){gpu=1;}}' |wc -l) | ||
cores_per_instance="$(lscpu |grep -E 'Core\(s\) per socket:|Socket\(s\):' |awk -v i="${xpu_num}" 'BEGIN{sum=1}{sum*=$NF}END{print sum/i}')" | ||
export OMP_NUM_THREADS=${cores_per_instance} | ||
for suite in $(echo ${{ inputs.suite }} |sed 's/,/ /g') | ||
do | ||
if [ "${suite}" == "pt2e" ];then | ||
continue | ||
fi | ||
contains "huggingface,timm_models,torchbench" $suite | ||
$contains_status | ||
for dt in $(echo ${{ inputs.dt }} |sed 's/,/ /g') | ||
do | ||
contains "float32,bfloat16,float16,amp_bf16,amp_fp16" $dt | ||
$contains_status | ||
for mode in $(echo ${{ inputs.mode }} |sed 's/,/ /g') | ||
do | ||
contains "inference,training" $mode | ||
$contains_status | ||
for scenario in $(echo ${{ inputs.scenario }} |sed 's/,/ /g') | ||
do | ||
contains "accuracy,performance" $scenario | ||
$contains_status | ||
if [ "${MODEL_ONLY_NAME}" == "" ];then | ||
for xpu_id in $(seq 0 $[ ${xpu_num} - 1 ]) | ||
do | ||
cpu_list="$(echo "${cores_per_instance} ${xpu_id}" |awk '{printf("%d-%d", $1*$2, $1*$2+$1-1)}')" | ||
numactl --localalloc --physcpubind=${cpu_list} bash -x inductor_xpu_test.sh ${suite} ${dt} ${mode} ${scenario} xpu ${xpu_id} static ${xpu_num} ${xpu_id} & | ||
done | ||
else | ||
for test_model in $(echo ${MODEL_ONLY_NAME} |sed 's/,/ /g') | ||
do | ||
numactl --localalloc bash -x inductor_xpu_test.sh ${suite} ${dt} ${mode} ${scenario} xpu 0 static 1 0 ${test_model} | ||
done | ||
fi | ||
wait | ||
# summarize pass rate | ||
LOG_DIR="inductor_log/${suite}/${dt}" | ||
LOG_NAME=inductor_${suite}_${dt}_${mode}_xpu_${scenario}_all.log | ||
rm -f ${LOG_DIR}/${LOG_NAME} | ||
find ${LOG_DIR}/ -name "inductor_${suite}_${dt}_${mode}_xpu_${scenario}_card*.log" |xargs cat >> ${LOG_DIR}/${LOG_NAME} 2>&1 | ||
done | ||
done | ||
done | ||
done | ||
|
||
- name: Summary E2E Test (${{ inputs.suite }} ${{ inputs.dt }} ${{ inputs.mode }} ${{ inputs.scenario }}) | ||
shell: bash -xe {0} | ||
run: | | ||
cd ./pytorch | ||
rm -f inductor_log/summary_accuracy.csv | ||
for var in $(find inductor_log/ -name "inductor_*_xpu_accuracy.csv") | ||
do | ||
sed -i "s/$/,$(basename $var)/" $var | ||
cat $var >> inductor_log/summary_accuracy.csv | ||
done | ||
cp ${{ github.workspace }}/.github/scripts/inductor_summary.py ./ | ||
csv_file="$(find inductor_log/ -name "inductor_*_xpu_*.csv" |tail -n 1)" | ||
if [ -f "${csv_file}" ];then | ||
pip install styleFrame scipy pandas | ||
dt=$(echo ${{ inputs.dt }} |sed 's/,/ /g') | ||
mode=$(echo ${{ inputs.mode }} |sed 's/,/ /g') | ||
suite=$(echo ${{ inputs.suite }} |sed 's/,/ /g') | ||
scenario=$(echo ${{ inputs.scenario }} |sed 's/,/ /g') | ||
python inductor_summary.py -p ${dt} -s ${suite} -m ${mode} -sc ${scenario} | ||
fi |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.