From 6b8e3b72939b14ac46103c7776be85a73afc3032 Mon Sep 17 00:00:00 2001 From: Scott Carda Date: Mon, 9 Mar 2026 13:57:35 -0700 Subject: [PATCH] Add a manual test plan notebook --- azure_quantum_manual_tests.ipynb | 401 +++++++++++++++++++++++++++++++ 1 file changed, 401 insertions(+) create mode 100644 azure_quantum_manual_tests.ipynb diff --git a/azure_quantum_manual_tests.ipynb b/azure_quantum_manual_tests.ipynb new file mode 100644 index 00000000..aa13d8f9 --- /dev/null +++ b/azure_quantum_manual_tests.ipynb @@ -0,0 +1,401 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "8b4edafb", + "metadata": {}, + "source": [ + "# Azure Quantum Manual Test Plan: Q# and Qiskit Job Submission\n", + "\n", + "This is a manual test plan notebook for verifying end-to-end job submission and result correctness against Azure Quantum.\n", + "\n", + "## What's Tested\n", + "- **Q# job submission** to three simulators via `target.submit()`\n", + "- **Qiskit job submission** to three simulators via `AzureQuantumProvider`\n", + "\n", + "## Simulators Under Test\n", + "\n", + "| Target | Provider |\n", + "|--------|----------|\n", + "| `ionq.simulator` | IonQ |\n", + "| `quantinuum.sim.h2-1e` | Quantinuum |\n", + "| `rigetti.sim.qvm` | Rigetti |\n", + "\n", + "## Test Circuit\n", + "All tests use a 3-qubit GHZ circuit, which produces the Bell state $\\frac{1}{\\sqrt{2}}(|000\\rangle + |111\\rangle)$. Valid results will show approximately 50% `000` and 50% `111` outcomes. Each test cell ends with an assertion that validates this.\n", + "\n", + "## How to Run\n", + "1. Fill in the workspace connection cell below with your Azure Quantum workspace details.\n", + "2. Run cells top to bottom within each test section.\n", + "3. Each job submission blocks until the job completes and prints a `PASS` or raises an `AssertionError`." + ] + }, + { + "cell_type": "markdown", + "id": "e5c54bbe", + "metadata": {}, + "source": [ + "## Setup\n", + "\n", + "Before opening this notebook, create a clean Python environment, install the local `azure-quantum` build into it, and point the notebook kernel at that environment:\n", + "\n", + "1. **Create a virtual environment:**\n", + " ```\n", + " python -m venv {env_name}\n", + " ```\n", + "\n", + "2. **Activate it:**\n", + " - Windows: `{env_name}\\Scripts\\activate`\n", + " - macOS/Linux: `source {env_name}/bin/activate`\n", + "\n", + "3. **Install the local `azure-quantum` package** (run from the root of the `azure-quantum-python` repo):\n", + " ```\n", + " pip install \".\\azure-quantum\\.[qsharp,qiskit]\"\n", + " ```\n", + "\n", + "4. **Set the kernel for this notebook** to `{env_name}` using the kernel picker in the top-right corner of the notebook editor." + ] + }, + { + "cell_type": "markdown", + "id": "1a4c6f9b", + "metadata": {}, + "source": [ + "## Workspace Configuration\n", + "\n", + "Set the `resource_id` below to point to your Azure Quantum workspace. This is the only cell you need to change when switching workspaces — all test sections below share this connection." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ff8f9336", + "metadata": {}, + "outputs": [], + "source": [ + "from azure.quantum import Workspace\n", + "\n", + "workspace = Workspace(\n", + " resource_id=\"\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "6a070d48", + "metadata": {}, + "source": [ + "---\n", + "\n", + "## Q# Job Tests\n", + "\n", + "Submit a compiled Q# program to each simulator target via `target.submit()`. Results are returned as a probability dictionary, e.g. `{'[0, 0, 0]': 0.47, '[1, 1, 1]': 0.53}`." + ] + }, + { + "cell_type": "markdown", + "id": "7350bd35", + "metadata": {}, + "source": [ + "Initialize `qsharp` with the `Base` target profile, define the GHZ operation in Q#, and compile it into a QIR payload that can be submitted directly to an Azure Quantum target." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2a5a78d7", + "metadata": {}, + "outputs": [], + "source": [ + "import qsharp\n", + "\n", + "qsharp.init(target_profile=qsharp.TargetProfile.Base)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cdb76f19", + "metadata": { + "vscode": { + "languageId": "qsharp" + } + }, + "outputs": [], + "source": [ + "%%qsharp\n", + "\n", + "operation GHZ() : Result[] {\n", + " use qs = Qubit[3];\n", + " H(qs[0]);\n", + " CNOT(qs[0], qs[1]);\n", + " CNOT(qs[1], qs[2]);\n", + " return MResetEachZ(qs);\n", + "}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bf14f04e", + "metadata": {}, + "outputs": [], + "source": [ + "GHZ = qsharp.compile(\"GHZ()\")" + ] + }, + { + "cell_type": "markdown", + "id": "827066bc", + "metadata": {}, + "source": [ + "Define the validation and submission helpers for the Q# section. `validate_qsharp_ghz` asserts that both `[0, 0, 0]` and `[1, 1, 1]` outcomes are present and each accounts for roughly half of the probability mass. `submit_qsharp_job` wraps `target.submit()` for a given target name." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "aaf1f865", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import cast\n", + "from azure.quantum.target import Target\n", + "\n", + "\n", + "def validate_qsharp_ghz(target_name, results, tolerance=0.35):\n", + " \"\"\"\n", + " Validate results from a Q# GHZ job submitted via target.submit().\n", + " Expected format: {'[0, 0, 0]': ~0.5, '[1, 1, 1]': ~0.5} (proportions, not counts).\n", + " \"\"\"\n", + " unexpected = {k: v for k, v in results.items() if k not in ('[0, 0, 0]', '[1, 1, 1]')}\n", + " if unexpected:\n", + " print(f\" [{target_name}] WARN: unexpected outcomes: {unexpected}\")\n", + " prob_000 = results.get('[0, 0, 0]', 0.0)\n", + " prob_111 = results.get('[1, 1, 1]', 0.0)\n", + " assert prob_000 > 0, f\"[{target_name}] Expected '[0, 0, 0]' in results, got: {results}\"\n", + " assert prob_111 > 0, f\"[{target_name}] Expected '[1, 1, 1]' in results, got: {results}\"\n", + " assert abs(prob_000 - 0.5) < tolerance, \\\n", + " f\"[{target_name}] [0,0,0] probability {prob_000:.2f} too far from 0.5 (tolerance {tolerance})\"\n", + " assert abs(prob_111 - 0.5) < tolerance, \\\n", + " f\"[{target_name}] [1,1,1] probability {prob_111:.2f} too far from 0.5 (tolerance {tolerance})\"\n", + " print(f\" [{target_name}] PASS: [0,0,0]={prob_000:.1%}, [1,1,1]={prob_111:.1%}\")\n", + "\n", + "\n", + "def submit_qsharp_job(target_name, shots=100):\n", + " \"\"\"Submit the compiled GHZ program to the given target and return the job.\"\"\"\n", + " target = cast(Target, workspace.get_targets(target_name))\n", + " return target.submit(GHZ, name=f\"ghz-qsharp-{target_name}\", shots=shots)" + ] + }, + { + "cell_type": "markdown", + "id": "76467ab2", + "metadata": {}, + "source": [ + "Submit the GHZ program to all three targets in a fast serial loop (submissions are non-blocking), then wait for all responses concurrently using a thread pool. Results are validated as they arrive." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f3d09585", + "metadata": {}, + "outputs": [], + "source": [ + "import concurrent.futures\n", + "\n", + "qsharp_targets = [\n", + " \"ionq.simulator\",\n", + " \"quantinuum.sim.h2-1e\",\n", + " \"rigetti.sim.qvm\",\n", + "]\n", + "\n", + "# Submit all jobs first (non-blocking)\n", + "print(\"Submitting Q# GHZ jobs...\")\n", + "jobs = {}\n", + "for target_name in qsharp_targets:\n", + " jobs[target_name] = submit_qsharp_job(target_name)\n", + " print(f\" Submitted to {target_name} (id: {jobs[target_name].id})\")\n", + "\n", + "# Wait for all results in parallel\n", + "print(\"\\nWaiting for results...\")\n", + "with concurrent.futures.ThreadPoolExecutor() as executor:\n", + " futures = {executor.submit(job.get_results): target_name for target_name, job in jobs.items()}\n", + " for future in concurrent.futures.as_completed(futures):\n", + " target_name = futures[future]\n", + " results = future.result()\n", + " print(f\" [{target_name}] Results: {results}\")\n", + " validate_qsharp_ghz(target_name, results)" + ] + }, + { + "cell_type": "markdown", + "id": "0cf9ca56", + "metadata": {}, + "source": [ + "---\n", + "\n", + "## Qiskit Job Tests\n", + "\n", + "Submit a Qiskit circuit to each simulator target via `AzureQuantumProvider`. Results come back as shot counts, e.g. `{'000': 47, '111': 53}`." + ] + }, + { + "cell_type": "markdown", + "id": "d2be5915", + "metadata": {}, + "source": [ + "Build the equivalent 3-qubit GHZ circuit in Qiskit, which will be submitted to each target backend below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a1cd1595", + "metadata": {}, + "outputs": [], + "source": [ + "from qiskit import QuantumCircuit\n", + "\n", + "# 3-qubit GHZ circuit matching the Q# GHZ operation above\n", + "circuit = QuantumCircuit(3, 3)\n", + "circuit.name = \"GHZ-3\"\n", + "circuit.h(0)\n", + "circuit.cx(0, 1)\n", + "circuit.cx(1, 2)\n", + "circuit.measure([0, 1, 2], [0, 1, 2])\n", + "circuit.draw(output=\"text\")" + ] + }, + { + "cell_type": "markdown", + "id": "f01f1964", + "metadata": {}, + "source": [ + "Connect to the workspace via `AzureQuantumProvider` and list the available backends to confirm the expected targets are accessible." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "202910f3", + "metadata": {}, + "outputs": [], + "source": [ + "from azure.quantum.qiskit import AzureQuantumProvider\n", + "\n", + "provider = AzureQuantumProvider(workspace)\n", + "print(\"Available backends:\")\n", + "for b in provider.backends():\n", + " print(\" -\", b.name)" + ] + }, + { + "cell_type": "markdown", + "id": "0b248dbf", + "metadata": {}, + "source": [ + "Define the validation and submission helpers for the Qiskit section. `validate_qiskit_ghz` asserts that both `000` and `111` shot counts are present and each accounts for roughly half of all shots. `submit_qiskit_job` wraps `backend.run()` for a given target name." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "86b7122a", + "metadata": {}, + "outputs": [], + "source": [ + "def validate_qiskit_ghz(target_name, counts, tolerance=0.35):\n", + " \"\"\"\n", + " Validate results from a Qiskit GHZ job submitted via AzureQuantumProvider.\n", + " Expected format: {'000': ~50%, '111': ~50%} of total shots.\n", + " \"\"\"\n", + " total = sum(counts.values())\n", + " assert total > 0, f\"[{target_name}] No results returned\"\n", + " unexpected = {k: v for k, v in counts.items() if k not in ('000', '111')}\n", + " if unexpected:\n", + " print(f\" [{target_name}] WARN: unexpected outcomes: {unexpected}\")\n", + " count_000 = counts.get('000', 0)\n", + " count_111 = counts.get('111', 0)\n", + " assert count_000 > 0, f\"[{target_name}] Expected '000' in counts, got: {counts}\"\n", + " assert count_111 > 0, f\"[{target_name}] Expected '111' in counts, got: {counts}\"\n", + " ratio_000 = count_000 / total\n", + " ratio_111 = count_111 / total\n", + " assert abs(ratio_000 - 0.5) < tolerance, \\\n", + " f\"[{target_name}] '000' ratio {ratio_000:.2f} too far from 0.5 (tolerance {tolerance})\"\n", + " assert abs(ratio_111 - 0.5) < tolerance, \\\n", + " f\"[{target_name}] '111' ratio {ratio_111:.2f} too far from 0.5 (tolerance {tolerance})\"\n", + " print(f\" [{target_name}] PASS: '000'={count_000} ({ratio_000:.1%}), '111'={count_111} ({ratio_111:.1%}), total={total}\")\n", + "\n", + "\n", + "def submit_qiskit_job(target_name, shots=100):\n", + " \"\"\"Submit the Qiskit GHZ circuit to the given backend and return the job.\"\"\"\n", + " backend = provider.get_backend(target_name)\n", + " return backend.run(circuit, shots=shots, job_name=f\"ghz-qiskit-{target_name}\")" + ] + }, + { + "cell_type": "markdown", + "id": "ef621c6f", + "metadata": {}, + "source": [ + "Submit the GHZ circuit to all three backends in a fast serial loop (submissions are non-blocking), then wait for all responses concurrently using a thread pool. Counts are validated as they arrive." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f1b660ce", + "metadata": {}, + "outputs": [], + "source": [ + "import concurrent.futures\n", + "\n", + "qiskit_targets = [\n", + " \"ionq.simulator\",\n", + " \"quantinuum.sim.h2-1e\",\n", + " \"rigetti.sim.qvm\",\n", + "]\n", + "\n", + "# Submit all jobs first (non-blocking)\n", + "print(\"Submitting Qiskit GHZ jobs...\")\n", + "jobs = {}\n", + "for target_name in qiskit_targets:\n", + " jobs[target_name] = submit_qiskit_job(target_name)\n", + " print(f\" Submitted to {target_name} (id: {jobs[target_name].job_id()})\")\n", + "\n", + "# Wait for all results in parallel\n", + "print(\"\\nWaiting for results...\")\n", + "with concurrent.futures.ThreadPoolExecutor() as executor:\n", + " futures = {executor.submit(job.result): target_name for target_name, job in jobs.items()}\n", + " for future in concurrent.futures.as_completed(futures):\n", + " target_name = futures[future]\n", + " counts = future.result().get_counts()\n", + " print(f\" [{target_name}] Counts: {counts}\")\n", + " validate_qiskit_ghz(target_name, counts)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}