Fix Docker Test Container Connection Failure (#852)

* add retries

* use windows

* run sequentially

* use windows

* output complete exception

* skip api version check

* pin azure-storage-blob

* revert other changes back

* set timezone before setting env variables

* output logs for failing tests

* remove timezone implementation changes

* configure automatic retries

* simplify retry implementation

Author: ahmedmuhsin

dockertests/azure-functions-test-kit/README.md

@@ -0,0 +1,121 @@
+# Azure Functions Test Kit
+
+Reusable test kit for testing Azure Functions workers in Docker containers.
+
+## Features
+
+### 🔄 Automatic Test Retries
+
+Flaky tests are automatically retried using [`pytest-rerunfailures`](https://github.com/pytest-dev/pytest-rerunfailures). This is especially useful for integration tests that may fail due to timing issues, network hiccups, or container startup delays.
+
+**Default behavior** (configured in `pyproject.toml` via `addopts`):
+- Tests are automatically retried up to **2 times** on failure
+- **5 seconds delay** between retry attempts
+
+**Override via command line:**
+
+```bash
+# Disable retries completely
+pytest --reruns 0
+
+# Set custom retry count
+pytest --reruns 5 --reruns-delay 3
+```
+
+**Override via environment variable (CI pipelines):**
+
+```bash
+# pytest's built-in env var for injecting CLI options
+export PYTEST_ADDOPTS="--reruns 3 --reruns-delay 10"
+```
+
+### 🐳 Automatic Container Logs on Failure
+
+When a test fails, the test kit automatically captures and displays the container logs. This helps with debugging by showing exactly what happened inside the container.
+
+**Example output:**
+```
+================================================================================
+🐳 CONTAINER LOGS FOR FAILED TEST
+================================================================================
+[2026-01-13 10:30:15] Starting Azure Functions host...
+[2026-01-13 10:30:16] Worker initialization started...
+[2026-01-13 10:30:17] Error: Failed to load function app
+================================================================================
+```
+
+### 📋 Environment Configuration
+
+The plugin automatically loads configuration from `.env` files in your test directory or parent directories (up to 3 levels).
+
+**Supported environment variables:**
+
+```env
+# Storage Configuration
+FUNCTIONS_TEST_USE_AZURITE=true                    # Use Azurite emulator (default: true)
+FUNCTIONS_TEST_STORAGE_CONNECTION_STRING=...       # Azure Storage connection string (for real storage)
+
+# Runtime Configuration
+FUNCTIONS_TEST_RUNTIME=java                        # Runtime: java, python, etc. (default: java)
+FUNCTIONS_TEST_RUNTIME_VERSION=21                  # Runtime version (default: 21)
+FUNCTIONS_TEST_HOST_VERSION=4                      # Functions host version (default: 4)
+
+# Test Configuration
+FUNCTIONS_TEST_APPS_DIR=./app-packages            # Directory with app packages (default: ./app-packages)
+FUNCTIONS_TEST_WORKER_DIR=/path/to/worker         # Custom worker directory to mount
+
+# Retry Configuration: use PYTEST_ADDOPTS or pyproject.toml addopts
+# See "Automatic Test Retries" section above
+```
+
+## Installation
+
+```bash
+pip install -e ./azure-functions-test-kit
+```
+
+The plugin is automatically activated via pytest's entry point mechanism.
+
+## Usage
+
+### Basic Test Example
+
+```python
+import pytest
+from azure_functions_test_kit import LinuxConsumptionTestEnvironment
+
+@pytest.fixture
+def test_env():
+    """Create a fresh test environment for each test"""
+    with LinuxConsumptionTestEnvironment(apps_to_upload=['MyApp']) as env:
+        yield env
+
+def test_my_function(test_env):
+    # Test automatically gets retries and log capture
+    response = requests.get(f"{test_env.url}/api/MyFunction")
+    assert response.status_code == 200
+```
+
+### Disabling Retries for Specific Tests
+
+```python
+@pytest.mark.no_rerun  # This test will not be retried
+def test_critical_function(test_env):
+    # This test will fail immediately without retries
+    pass
+```
+
+## How It Works
+
+1. **Plugin Discovery**: pytest automatically discovers the plugin via the `pytest11` entry point
+2. **Environment Loading**: `.env` files are loaded during `pytest_configure`
+3. **Retry Configuration**: `pytest-rerunfailures` handles retries via `addopts` in `pyproject.toml`
+4. **Test Execution**: Tests run with automatic retry on failure
+5. **Log Capture**: On failure, container logs are captured and displayed
+
+## Benefits
+
+- **Reduced flakiness**: Automatic retries handle transient failures
+- **Better debugging**: Container logs are immediately available on failure
+- **Flexible configuration**: Override defaults via environment variables or command line
+- **Zero code changes**: Works automatically with existing tests

dockertests/azure-functions-test-kit/pyproject.toml

@@ -10,7 +10,8 @@ requires-python = ">=3.8"
 dependencies = [
     "pytest>=7.0",
     "pytest-xdist>=3.0",
-    "azure-storage-blob>=12.19.0",
+    "pytest-rerunfailures>=12.0",
+    "azure-storage-blob==12.27.1",
     "requests>=2.31.0",
     "cryptography>=41.0.0",
     "python-dotenv>=1.0.0",

dockertests/azure-functions-test-kit/src/azure_functions_test_kit/plugin.py

@@ -6,9 +6,9 @@
 Provides automatic .env file loading and optional helper fixtures.
 Tests are free to create their own fixtures using the exported classes.
 """
-import os
 from pathlib import Path
 from dotenv import load_dotenv
+import pytest
 
 
 def pytest_configure(config):
@@ -30,3 +30,42 @@ def pytest_configure(config):
                 break
         else:
             print("ℹ️  [azure-functions-test-kit] No .env file found, using environment variables")
+
+
+@pytest.hookimpl(tryfirst=True, hookwrapper=True)
+def pytest_runtest_makereport(item, call):
+    """Hook to capture test failures and display container logs."""
+    # Execute all other hooks to obtain the report object
+    outcome = yield
+    rep = outcome.get_result()
+    
+    # Only process test failures in the call phase (actual test execution)
+    if rep.when == "call" and rep.failed:
+        # Try to get the test_env fixture from the test
+        test_env = None
+        if hasattr(item, 'funcargs') and 'test_env' in item.funcargs:
+            test_env = item.funcargs['test_env']
+        
+        if test_env and hasattr(test_env, 'functions_controller'):
+            functions_controller = test_env.functions_controller
+            if functions_controller:
+                try:
+                    logs = functions_controller.get_container_logs()
+                    
+                    # Add container logs to the test report
+                    print("\n" + "="*80)
+                    print("🐳 CONTAINER LOGS FOR FAILED TEST")
+                    print("="*80)
+                    print(logs)
+                    print("="*80 + "\n")
+                    
+                    # Also append to the test's longrepr for visibility in reports
+                    if hasattr(rep, 'longrepr'):
+                        log_section = f"\n\n{'='*80}\n🐳 CONTAINER LOGS\n{'='*80}\n{logs}\n{'='*80}\n"
+                        if rep.longrepr:
+                            rep.longrepr = str(rep.longrepr) + log_section
+                        else:
+                            rep.longrepr = log_section
+                            
+                except Exception as e:
+                    print(f"\n⚠️  Failed to retrieve container logs: {e}\n")

dockertests/pyproject.toml

@@ -25,4 +25,4 @@ testpaths = ["."]
 pythonpath = ["."]
 python_files = ["test_*.py"]
 python_classes = ["Test*", "!TestEnvironment"]
-addopts = ["-v", "--tb=short"]
+addopts = ["-v", "--tb=short", "--reruns=2", "--reruns-delay=5"]

eng/ci/templates/jobs/run-docker-tests-linux.yml

@@ -62,7 +62,7 @@ jobs:
 
       - bash: |
           cd dockertests
-          pytest . -v -n auto --tb=short
+          pytest . -v -n auto --tb=long
         displayName: 'Run Docker integration tests'
         env:
           FUNCTIONS_TEST_WORKER_DIR: '$(Build.SourcesDirectory)/java-worker'