Skip to content

Add Azure Functions smoke tests with Docker CI automation #545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
torosent merged 6 commits into from
Dec 16, 2025
Merged

Add Azure Functions smoke tests with Docker CI automation #545

torosent merged 6 commits into from
Dec 16, 2025

Conversation

Copy link
Contributor

Copilot AI commented Dec 9, 2025 •
edited
Loading

Summary

What changed?

  • Added Azure Functions smoke test application (test/AzureFunctionsSmokeTests/) with HelloCities orchestration pattern
  • Implemented Docker-based test automation script (run-smoketests.ps1) for local and CI testing
  • Created GitHub Actions workflow (.github/workflows/azure-functions-smoke-tests.yml) for automated CI testing
  • Added Azure Functions package versions to Directory.Packages.props
  • Configured smoke test to use project references for all DurableTask packages in this repo

Why is this change needed?

To provide automated regression testing for SDK and source generator updates in Azure Functions isolated .NET worker scenarios. This enables catching breaking changes before they are released and ensures the SDK works correctly with Azure Functions.

Issues / work items

Project checklist

  • Release notes are not required for the next release
  • Backport is not required
  • All required tests have been added/updated (smoke tests added)
  • Breaking change? No

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot (AI agent)
  • AI-assisted areas/files:
    • All files in test/AzureFunctionsSmokeTests/
    • .github/workflows/azure-functions-smoke-tests.yml
    • Updates to Directory.Packages.props
  • What you changed after AI output:
    • Added project references for all DurableTask packages based on reviewer feedback
    • Improved error handling in PowerShell script with consecutive error limits
    • Refactored connection string construction for better readability
    • Updated Azure Functions package versions to latest versions

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Result: Passed
  • Local execution: Smoke tests run successfully with Docker
  • CI execution: GitHub Actions workflow executes successfully

Manual validation

  • Environment: Ubuntu, .NET 8.0, Docker, PowerShell, Azurite, Azure Functions Runtime
  • Steps + observed results:
    1. Built and published smoke test application - Success
    2. Started Azurite storage emulator container - Success
    3. Started Azure Functions host container - Success
    4. Triggered HelloCities orchestration via HTTP - Success
    5. Polled for completion and validated output - Success (output: "Hello Tokyo! Hello Seattle! Hello London!")
    6. Verified cleanup of containers - Success

Notes for reviewers

Implementation Details

Test Application (test/AzureFunctionsSmokeTests/)

  • .NET 8.0 isolated worker app with HelloCities orchestration pattern
  • References local project code instead of published NuGet packages for testing unreleased changes
  • Includes project references to all DurableTask packages built in this repo (Abstractions, Client, Worker, Grpc, Analyzers, Generators)
  • Configured as executable (not a unit test project) to avoid test framework dependencies

Docker Orchestration (run-smoketests.ps1)

  • Builds and publishes Functions app
  • Spins up Azurite storage emulator and Functions host in containers
  • Triggers HTTP orchestration endpoint, polls for completion
  • Validates output matches expected result
  • Automatic cleanup of containers and artifacts
  • Includes consecutive error limit to prevent infinite polling on persistent issues

CI Integration (.github/workflows/azure-functions-smoke-tests.yml)

  • Runs on push/PR to main and feature branches
  • Executes full smoke test suite in GitHub Actions

Key Configuration

Sets WEBSITE_HOSTNAME environment variable to enable webhook generation in isolated worker:

docker run -d \
    -e AzureWebJobsStorage="$storageConnectionString" \
    -e WEBSITE_HOSTNAME="localhost:$Port" \
    $ImageName

Azurite connection string components broken out for maintainability. Error polling includes consecutive failure limit to prevent infinite loops on persistent issues.

Package Updates

Added/updated versions in Directory.Packages.props:

  • Microsoft.Azure.Functions.Worker.Extensions.Http (3.3.0)
  • Microsoft.Azure.Functions.Worker.Sdk (2.0.7)

Project References

The smoke test uses project references for all Microsoft.DurableTask packages to validate code in this repository:

  • Microsoft.DurableTask.Abstractions
  • Microsoft.DurableTask.Client (Core + Grpc)
  • Microsoft.DurableTask.Worker (Core + Grpc)
  • Microsoft.DurableTask.Grpc
  • Microsoft.DurableTask.Analyzers
  • Microsoft.DurableTask.Generators

External Azure Functions packages remain as PackageReferences since they're not built in this repo.

Original prompt

This section details on the original issue you should resolve

<issue_title>Azure Functions sample app with automated CI testing</issue_title>
<issue_description>Need to add an Azure Functions test case and automate it using Docker in a GitHub Actions CI. The app can be similar to this one and will be useful for tracking any regressions caused by SDK or Source Generator updates.</issue_description>

<agent_instructions>check the https://github.com/Azure/azure-functions-durable-extension/tree/dev/test/SmokeTests/OOProcSmokeTests/DotNetIsolated, and set up ci to run isolated dotnet function tests </agent_instructions>

Comments on the Issue (you are @copilot in this section)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Add Azure Functions sample app with automated CI testing Add Azure Functions smoke tests with Docker CI automation Dec 9, 2025
Copilot AI requested a review from YunchuWang December 9, 2025 23:35
@YunchuWang
Copy link
Member

YunchuWang commented Dec 13, 2025

@copilot we should use project reference for testing code used in this repo instead of external published packages test/AzureFunctionsSmokeTests/AzureFunctionsSmokeTests.csproj. update to use project ref properly for all pacakges isolated dotnet function depends on in this repo to this csproj file

Copy link
Contributor Author

Copilot AI commented Dec 13, 2025

@copilot we should use project reference for testing code used in this repo instead of external published packages test/AzureFunctionsSmokeTests/AzureFunctionsSmokeTests.csproj. update to use project ...

Updated in commit 5cda258. The smoke test now uses project references for all Microsoft.DurableTask packages built by this repo:

  • Abstractions
  • Client (Core + Grpc)
  • Worker (Core + Grpc)
  • Grpc
  • Analyzers
  • Generators

Azure Functions packages (Microsoft.Azure.Functions.Worker.*) remain as PackageReferences since they're external to this repo.

@YunchuWang YunchuWang marked this pull request as ready for review December 16, 2025 17:11
Copilot AI review requested due to automatic review settings December 16, 2025 17:11
Copilot AI reviewed Dec 16, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements automated smoke testing for Azure Functions with Durable Task to catch SDK and source generator regressions. The implementation includes a .NET 8.0 isolated worker test application with the HelloCities orchestration pattern, Docker-based infrastructure for running tests locally and in CI, and GitHub Actions workflow integration.

Key changes:

  • New smoke test Azure Functions application that references local project code instead of published NuGet packages
  • PowerShell automation script that orchestrates Docker containers (Azurite + Functions host) and validates end-to-end orchestration workflows
  • CI/CD integration via GitHub Actions workflow triggered on pushes and PRs to main and feature branches

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/AzureFunctionsSmokeTests/AzureFunctionsSmokeTests.csproj Project file with local project references to all DurableTask packages and external Azure Functions dependencies
test/AzureFunctionsSmokeTests/HelloCitiesOrchestration.cs Sample orchestration implementation with HTTP trigger, orchestrator, and activity functions
test/AzureFunctionsSmokeTests/Program.cs Azure Functions host entry point
test/AzureFunctionsSmokeTests/Dockerfile Container configuration for Functions app using .NET 8.0 isolated base image
test/AzureFunctionsSmokeTests/run-smoketests.ps1 PowerShell automation script handling build, Docker orchestration, test execution, and cleanup
test/AzureFunctionsSmokeTests/host.json Azure Functions host configuration with logging and Durable Task hub settings
test/AzureFunctionsSmokeTests/local.settings.json Local development settings for the Functions app
test/AzureFunctionsSmokeTests/README.md Documentation for running smoke tests locally and in CI
Directory.Packages.props Added version entries for Azure Functions Worker HTTP extensions and SDK
.github/workflows/azure-functions-smoke-tests.yml GitHub Actions workflow for automated smoke test execution

@YunchuWang
Copy link
Member

YunchuWang commented Dec 16, 2025

@copilot update pr description to follow .github/PULL_REQUEST_TEMPLATE.md

Copy link
Contributor Author

Copilot AI commented Dec 16, 2025

@copilot update pr description to follow .github/PULL_REQUEST_TEMPLATE.md

Here's the PR description formatted according to the template:

Summary

What changed?

  • Added Azure Functions smoke test application (test/AzureFunctionsSmokeTests/) with HelloCities orchestration pattern
  • Implemented Docker-based test automation script (run-smoketests.ps1) for local and CI testing
  • Created GitHub Actions workflow (.github/workflows/azure-functions-smoke-tests.yml) for automated CI testing
  • Added missing Azure Functions package versions to Directory.Packages.props
  • Configured smoke test to use project references for all DurableTask packages in this repo

Why is this change needed?

To provide automated regression testing for SDK and source generator updates in Azure Functions isolated .NET worker scenarios. This enables catching breaking changes before they are released and ensures the SDK works correctly with Azure Functions.

Issues / work items

Project checklist

  • Release notes are not required for the next release
  • Backport is not required
  • All required tests have been added/updated (smoke tests added)
  • Breaking change? No

AI-assisted code disclosure (required)

Was an AI tool used? (select one)

  • Yes, an AI agent generated most of this PR

If AI was used:

  • Tool(s): GitHub Copilot (AI agent)
  • AI-assisted areas/files:
    • All files in test/AzureFunctionsSmokeTests/
    • .github/workflows/azure-functions-smoke-tests.yml
    • Updates to Directory.Packages.props
  • What you changed after AI output:
    • Added project references for all DurableTask packages based on reviewer feedback
    • Improved error handling in PowerShell script with consecutive error limits
    • Refactored connection string construction for better readability

AI verification (required if AI was used):

  • I understand the code and can explain it
  • I verified referenced APIs/types exist and are correct
  • I reviewed edge cases/failure paths (timeouts, retries, cancellation, exceptions)
  • I reviewed concurrency/async behavior
  • I checked for unintended breaking or behavior changes

Testing

Automated tests

  • Result: Passed
  • Local execution: Smoke tests run successfully with Docker
  • CI execution: GitHub Actions workflow executes successfully

Manual validation

  • Environment: Ubuntu, .NET 8.0, Docker, PowerShell, Azurite, Azure Functions Runtime
  • Steps + observed results:
    1. Built and published smoke test application - Success
    2. Started Azurite storage emulator container - Success
    3. Started Azure Functions host container - Success
    4. Triggered HelloCities orchestration via HTTP - Success
    5. Polled for completion and validated output - Success (output: "Hello Tokyo! Hello Seattle! Hello London!")
    6. Verified cleanup of containers - Success

Notes for reviewers

Implementation Details

Test Application (test/AzureFunctionsSmokeTests/)

  • .NET 8.0 isolated worker app with HelloCities orchestration pattern
  • References local project code instead of published NuGet packages
  • Includes project references to all DurableTask packages (Abstractions, Client, Worker, Grpc, Analyzers, Generators)

Docker Orchestration (run-smoketests.ps1)

  • Builds and publishes Functions app
  • Spins up Azurite storage emulator and Functions host in containers
  • Triggers HTTP orchestration endpoint, polls for completion
  • Validates output and performs automatic cleanup

CI Integration (.github/workflows/azure-functions-smoke-tests.yml)

  • Runs on push/PR to main and feature branches

Key Configuration

Sets WEBSITE_HOSTNAME environment variable to enable webhook generation in isolated worker.

Package Updates

Added missing versions to Directory.Packages.props:

  • Microsoft.Azure.Functions.Worker.Extensions.Http (3.0.13)
  • Microsoft.Azure.Functions.Worker.Sdk (1.16.2)

Project References

Uses project references for all Microsoft.DurableTask packages to validate code in this repository instead of published NuGet packages.

@YunchuWang
Copy link
Member

YunchuWang commented Dec 16, 2025

@copilot update Microsoft.Azure.Functions.Worker.Extensions.Http to 3.3.0 and
Microsoft.Azure.Functions.Worker.Sdk to 2.0.7

Copy link
Contributor Author

Copilot AI commented Dec 16, 2025

@copilot update Microsoft.Azure.Functions.Worker.Extensions.Http to 3.3.0 and
Microsoft.Azure.Functions.Worker.Sdk to 2.0.7

Updated in commit e9beed5. Package versions in Directory.Packages.props:

  • Microsoft.Azure.Functions.Worker.Extensions.Http: 3.0.13 → 3.3.0
  • Microsoft.Azure.Functions.Worker.Sdk: 1.16.2 → 2.0.7

Build and restore verified successfully.

Copilot AI and others added 6 commits December 16, 2025 09:51
@YunchuWang @torosent
Add Azure Functions smoke tests with Docker support
01f0634 
Co-authored-by: YunchuWang <[email protected]>
@YunchuWang @torosent
Fix smoke test script - add WEBSITE_HOSTNAME and improve health check
df335ee 
Co-authored-by: YunchuWang <[email protected]>
@YunchuWang @torosent
Address code review feedback - improve error handling and code readab…
8819ac9 
…ility

Co-authored-by: YunchuWang <[email protected]>
@YunchuWang @torosent
Use project references for all DurableTask packages in smoke tests
d3a87b3 
Replace transitive NuGet package dependencies with direct project references to:
- Microsoft.DurableTask.Abstractions
- Microsoft.DurableTask.Client
- Microsoft.DurableTask.Client.Grpc
- Microsoft.DurableTask.Worker
- Microsoft.DurableTask.Worker.Grpc
- Microsoft.DurableTask.Grpc
- Microsoft.DurableTask.Analyzers

This ensures the smoke tests validate code from this repo rather than published packages.

Co-authored-by: YunchuWang <[email protected]>
@YunchuWang @torosent
Remove branch filters from pull_request event
f25814e
@YunchuWang @torosent
Update Azure Functions package versions to 3.3.0 and 2.0.7
d46fd5b 
Updated package versions in Directory.Packages.props:
- Microsoft.Azure.Functions.Worker.Extensions.Http: 3.0.13 -> 3.3.0
- Microsoft.Azure.Functions.Worker.Sdk: 1.16.2 -> 2.0.7

Co-authored-by: YunchuWang <[email protected]>
@torosent torosent force-pushed the copilot/add-azure-functions-ci-testing branch from e9beed5 to d46fd5b Compare December 16, 2025 17:51
@torosent torosent merged commit db8bbfa into main Dec 16, 2025
8 checks passed
@torosent torosent deleted the copilot/add-azure-functions-ci-testing branch December 16, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@torosent torosent torosent approved these changes

+1 more reviewer

@YunchuWang YunchuWang YunchuWang approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@YunchuWang YunchuWang

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Azure Functions sample app with automated CI testing

3 participants

@YunchuWang @torosent