Initial implementation of ASN.1 API #13325

facutuesca · 2025-08-20T00:20:33Z

Overview

This PR adds the initial structure for the ASN.1 module, plus support for defining SEQUENCE types and using INTEGER fields. Only the encoding logic is included, to keep the PR size small and to keep the focus on the structure of the code.

Here's an example of what using the API looks like:

import cryptography.hazmat.asn1 as asn1

@asn1.sequence
class Example:
    foo: int
    bar: int

value = Example(foo=9, bar=6)

encoded = asn1.encode_der(value)
assert encoded == b"\x30\x06\x02\x01\x09\x02\x01\x06"

Structure

The code is divided into a hazmat.asn1 Python module, and a Rust _rust.asn1_exp Rust module. This last one has a different name in order to avoid mixing it with the existing asn1 Rust module (which can be done later, but for now doing it like this simplifies the PR).

Details

Taking as an example the code snippet above, this is what happens:

While interpreting the class definition, the @asn1.sequence decorator is executed, which goes through the Example class and recursively reads its type annotations. This information is converted to a Rust object (AnnotatedType) and stored in the Example.__asn1_root__ and Example.__asn1_fields__ attributes. All of this happens in hazmat/asn1/asn1.py
An Example object is instantiated with some values, and asn1.encode_der is called on it. asn1.encode_der is a Rust function defined in asn1_exp/asn1.rs.
This function combines the type information (AnnotatedType object) and the value of the object into an AnnotatedTypeObject value, and encodes it using rust-asn1's asn1::write API.
This is possible because AnnotatedTypeObject implements the asn1::Asn1Writable trait. This is done in asn1_exp/encoding.rs
And that's it, the resulting bytes are now available in Python

Some things (like the empty Annotation struct) are there as placeholders for missing features I removed to keep this PR small. I removed their contents but left the empty struct so that the rest of the data and logic that depend on it will keep the same structure once we re-add the missing features.

facutuesca · 2025-08-20T01:12:31Z

I added typing-extensions as an unconditional dependency, but it can probably be removed for newer Python versions if we use typing when we detect a new-enough Python version

alex · 2025-08-20T01:52:01Z

looks like there's some missed coverage here -- do you want to take a look at the missing test cases before I review?

alex · 2025-08-20T01:54:41Z

src/cryptography/hazmat/asn1/asn1.py

+
+import typing_extensions as te
+
+from cryptography.hazmat.bindings._rust import asn1_exp


I don't love this module name, I'm not sure what exp even stands for :-)

ah lol, experimental, it was just meant to have a different name from the existing asn1 module

We probably need to find a real name 😂

I'm open to suggestions. asn1_api maybe?

I suggest declarative_asn1 for now since it provides the declarative ASN.1 API (or decl_asn1 if we need it to be short)

lets use the full word, bytes are free

src/cryptography/hazmat/asn1/asn1.py

alex · 2025-08-20T01:55:22Z

src/cryptography/hazmat/asn1/asn1.py

+from typing import (
+    Any,
+    ClassVar,
+    TypeVar,
+)


Similarly, house style is to reference these as typing. for clarity (Any is particularly unclear without the typing.)

src/cryptography/hazmat/asn1/asn1.py

alex · 2025-08-20T01:56:14Z

src/cryptography/hazmat/asn1/asn1.py

+) -> asn1_exp.AnnotatedType:
+    annotation = asn1_exp.Annotation()
+
+    if field_type == builtins.int:


Prefer is when comparing against types

alex · 2025-08-20T01:57:11Z

src/cryptography/hazmat/asn1/asn1.py

+
+def _register_asn1_type(cls: type[U], root_type: asn1_exp.RootType) -> None:
+    raw_fields = te.get_type_hints(cls, include_extras=True)
+    setattr(cls, "__asn1_fields__", _annotate_fields(raw_fields))


similarly: cls.__asn1_fields__ = _annotate_fields(raw_fields)

mypy complains:

src/cryptography/hazmat/asn1/asn1.py:55: error: "type[U]" has no attribute "__asn1_fields__" [attr-defined]

Hmm, this error doesn't make much sense to me.

Why do we need both __asn1_fields__ and __asn1_root__? It seems like only one should be required.

yup, see below, I removed __asn1_fields__

src/cryptography/hazmat/asn1/asn1.py

src/cryptography/hazmat/asn1/__init__.py

src/rust/src/asn1_exp/types.rs

pyproject.toml

facutuesca · 2025-08-20T22:13:14Z

looks like there's some missed coverage here -- do you want to take a look at the missing test cases before I review?

Missing tests added!

alex · 2025-08-21T02:14:22Z

Will review again tomorrow. I should have said at the top: this looks like a solid start!

alex · 2025-08-22T01:00:04Z

src/cryptography/hazmat/asn1/asn1.py

+
+def _register_asn1_type(cls: type[U], root_type: asn1_exp.RootType) -> None:
+    raw_fields = te.get_type_hints(cls, include_extras=True)
+    setattr(cls, "__asn1_fields__", _annotate_fields(raw_fields))


Hmm, this error doesn't make much sense to me.

alex · 2025-08-22T01:00:37Z

src/cryptography/hazmat/asn1/asn1.py

+
+def _register_asn1_type(cls: type[U], root_type: asn1_exp.RootType) -> None:
+    raw_fields = te.get_type_hints(cls, include_extras=True)
+    setattr(cls, "__asn1_fields__", _annotate_fields(raw_fields))


Why do we need both __asn1_fields__ and __asn1_root__? It seems like only one should be required.

alex · 2025-08-22T01:02:16Z

src/rust/src/asn1_exp/asn1.rs

+// 2.0, and the BSD License. See the LICENSE file in the root of this repository
+// for complete details.
+
+use pyo3::prelude::*;


We don't use import *

alex · 2025-08-22T01:02:43Z

src/rust/src/asn1_exp/encode.rs

+
+use crate::asn1_exp::types::{AnnotatedType, AnnotatedTypeObject, Type};
+
+fn write_value<T: SimpleAsn1Writable>(


This doesn't seem to do very much?

yeah, it's because it's useful to avoid duplication only when all the other types are implemented. See here (it saves us writing that match statement once for each type)

alex · 2025-08-22T01:03:11Z

src/rust/src/asn1_exp/encode.rs

+            Type::Sequence(cls) => {
+                let fields = cls
+                    .getattr(py, "__asn1_fields__")
+                    .map_err(|_| asn1::WriteError::AllocationError)?;


It seems like instead of a separate __asn1_fields__ we could just store the field types in Type::Sequence()?

yup, good idea. Now Type::Sequence contains the root type and the fields.

src/rust/src/asn1_exp/types.rs

alex · 2025-08-22T01:05:07Z

tests/hazmat/asn1/test_asn1.py

+                foo: Invalid
+
+
+class TestEncoding:


Let's put tests of specific types in their own test file.

alex · 2025-08-22T01:05:43Z

ci-constraints-requirements.txt

-    #   mypy
-    #   virtualenv
-typing-extensions==4.14.1 ; python_full_version >= '3.9'
+typing-extensions==4.13.2


This file should be built in a way that uses the highest version for each python version:

uv pip compile -U --universal --python-version 3.8 --all-extras --unsafe-package=cffi --unsafe-package=pycparser --unsafe-package=setuptools --unsafe-package=cryptography-vectors --unsafe-package=bcrypt pyproject.toml -o ci-constraints-requirements.txt -q is the right way to build it.

alex · 2025-08-22T01:05:56Z

pyproject.toml


    # Must be kept in sync with `project.dependencies`
    "cffi>=1.14; platform_python_implementation != 'PyPy'",
+    "typing-extensions>=4.13.2",


Why is this needed at build time?

it's not, ha. Fixed

alex · 2025-08-24T21:29:43Z

Looks like there's some merge conflicts

facutuesca · 2025-08-24T21:41:16Z

Looks like there's some merge conflicts

fixed!

alex

Two small comments, but otherwise this LGTM. I'd like @reaperhulk to have a look before we merge though.

alex · 2025-08-25T00:05:28Z

src/cryptography/hazmat/asn1/asn1.py

+
+    setattr(cls, "__asn1_root__", root)
+
+    def new_init(self: U, /, **kwargs: object) -> None:


We should probably codegen this for performance, but doesn't have to happen in this PR. Can you add a TODO though?

added a TODO comment

alex · 2025-08-25T00:08:14Z

tests/hazmat/asn1/test_encoding.py

+import cryptography.hazmat.asn1 as asn1
+
+
+class TestEncoding:


One class per type we're testing teh encoding of (sequence vs. int so far)

reaperhulk · 2025-08-25T15:22:16Z

pyproject.toml

    # Must be kept in sync with `build-system.requires`
    "cffi>=1.14; platform_python_implementation != 'PyPy'",
+    # Must be kept in sync with ./.github/requirements/build-requirements.{in,txt}
+    "typing-extensions>=4.13.2",


This is required because we need two functions that were added in Python 3.11 right? is it a requirement for >= 3.11?

If it's not, let's use the python_version branch

Looks like typing has this in 3.11+ (and Facundo did note that in a comment 5 days ago, but there's a lot of traffic on this PR 😄). Let's call it from typing where possible and set the python_version on the rest, so we know when we can remove this...in 4 years.

fixed! there was an extra typing function which we need to use typing-extensions for: typing.get_type_hints which we call with its include_extras parameter. That parameter was added in Python 3.9, so I also added it to the conditional.

Now it will use the typing-extensions variant for any Python < 3.11. I can make it even more precise (only use it for Python < 3.9), but for now I only left a comment saying to remove it when the min version is 3.9.

if sys.version_info < (3, 11): import typing_extensions # We use the `include_extras` parameter of `get_type_hints`, which was # added in Python 3.9, so this can be replaced by the `typing` version # once the min version is >= 3.9 get_type_hints = typing_extensions.get_type_hints else: get_type_hints = typing.get_type_hints

Is there a reason not to make this branch on 3.9? Seems cleaner.

nope, fixed

alex

A small handful of comments, but otherwise LGTM!

alex · 2025-08-27T01:14:28Z

src/cryptography/hazmat/asn1/asn1.py

+
+    @typing_extensions.dataclass_transform(kw_only_default=True)
+    def sequence(cls: type[U]) -> type[U]:
+        dataclass_cls = dataclasses.dataclass(cls)


We don't want all the defaults that dataclass has, I think to start with the only thing we want to define is init, everything else should be false.

Fixed. I had to add another Python version check, since one of the dataclass arguments (match_args) that defaults to True was added in Python 3.10.

alex · 2025-08-27T01:15:27Z

src/rust/src/declarative_asn1/types.rs

+#[pyo3::pyclass(frozen, module = "cryptography.hazmat.bindings._rust.asn1")]
+#[derive(Debug)]
+pub struct AnnotatedType {
+    #[pyo3(get)]


is there a reason to expose these?

Removed it for now, it will be needed in the future

alex · 2025-08-27T01:15:55Z

src/rust/src/declarative_asn1/types.rs

+#[pyo3::pymethods]
+impl Annotation {
+    #[new]
+    #[pyo3(signature = ())]


I don't think this is required?

src/cryptography/hazmat/asn1/asn1.py

alex

will wait for @reaperhulk to confirm he's goood, but I think this is ready

reaperhulk · 2025-09-13T07:44:03Z

This needs a rebase now, sorry!

Signed-off-by: Facundo Tuesca <[email protected]>

facutuesca marked this pull request as draft August 20, 2025 00:26

facutuesca force-pushed the ft/asn1-api-init branch 3 times, most recently from 077a337 to df5289a Compare August 20, 2025 01:07

facutuesca marked this pull request as ready for review August 20, 2025 01:21

alex reviewed Aug 20, 2025

View reviewed changes

alex reviewed Aug 22, 2025

View reviewed changes

facutuesca force-pushed the ft/asn1-api-init branch from 2cae2ee to 9ecdbe3 Compare August 24, 2025 21:39

alex reviewed Aug 25, 2025

View reviewed changes

reaperhulk reviewed Aug 25, 2025

View reviewed changes

alex reviewed Aug 27, 2025

View reviewed changes

facutuesca force-pushed the ft/asn1-api-init branch 4 times, most recently from 0e3bcc0 to aa6d03e Compare August 27, 2025 11:55

alex reviewed Aug 28, 2025

View reviewed changes

src/cryptography/hazmat/asn1/asn1.py Show resolved Hide resolved

facutuesca force-pushed the ft/asn1-api-init branch 5 times, most recently from dd06339 to 00cacb8 Compare August 30, 2025 14:02

alex previously approved these changes Aug 30, 2025

View reviewed changes

reaperhulk previously approved these changes Sep 13, 2025

View reviewed changes

facutuesca added 22 commits September 13, 2025 11:25

Remove unnecessary getattr

2a81400

Signed-off-by: Facundo Tuesca <[email protected]>

Remove unused RootType

b1c4d25

Signed-off-by: Facundo Tuesca <[email protected]>

Remove more import aliases

2f36c51

Signed-off-by: Facundo Tuesca <[email protected]>

Add test for missing coverage

2cc2288

Signed-off-by: Facundo Tuesca <[email protected]>

Remove unneeded build dependency

62f5d07

Signed-off-by: Facundo Tuesca <[email protected]>

Regenerate ci constraints requirements

7347fcf

Signed-off-by: Facundo Tuesca <[email protected]>

Deduplicate conversion of builtin Python types

cbc30f8

Signed-off-by: Facundo Tuesca <[email protected]>

Tidy up Rust imports

eebcf14

Signed-off-by: Facundo Tuesca <[email protected]>

Separate tests into files per test type

2bc7b68

Signed-off-by: Facundo Tuesca <[email protected]>

Remove unneeded __asn1_fields__ attribute

caf6c8d

Signed-off-by: Facundo Tuesca <[email protected]>

Re-add typing-extensions to build-requirements.in

5692c9f

Signed-off-by: Facundo Tuesca <[email protected]>

Split tests by ASN.1 type

503e1da

Signed-off-by: Facundo Tuesca <[email protected]>

Add TODO

47f959d

Signed-off-by: Facundo Tuesca <[email protected]>

Rename Rust module to declarative_asn1

b58f83f

Signed-off-by: Facundo Tuesca <[email protected]>

Use typing-extensions only for Python < 3.11

a81b245

Signed-off-by: Facundo Tuesca <[email protected]>

Use typing_extensions.get_type_hints only when <3.9

7393cf2

Signed-off-by: Facundo Tuesca <[email protected]>

Add comment explaining mypy workaround

f177c48

Signed-off-by: Facundo Tuesca <[email protected]>

Use dataclasses.dataclass to generate init method

69becf4

Signed-off-by: Facundo Tuesca <[email protected]>

Remove extra dataclass behavior

dc0826d

Signed-off-by: Facundo Tuesca <[email protected]>

Remove unused pyo3 annotations

7957bae

Signed-off-by: Facundo Tuesca <[email protected]>

Use kw_only with dataclasses

262a74b

Signed-off-by: Facundo Tuesca <[email protected]>

Add test for kw-only initialization

871834e

Signed-off-by: Facundo Tuesca <[email protected]>

facutuesca dismissed stale reviews from reaperhulk and alex via 871834e September 13, 2025 09:30

facutuesca force-pushed the ft/asn1-api-init branch from 00cacb8 to 871834e Compare September 13, 2025 09:30

reaperhulk approved these changes Sep 13, 2025

View reviewed changes

reaperhulk merged commit 0fb8f93 into pyca:main Sep 13, 2025
132 checks passed

facutuesca deleted the ft/asn1-api-init branch September 13, 2025 09:44

facutuesca mentioned this pull request Sep 16, 2025

Add decoding support to ASN.1 API #13449

Merged

facutuesca mentioned this pull request Oct 20, 2025

[META] Declarative API for ASN.1 in Python #12283

Open


		import typing_extensions as te

		from cryptography.hazmat.bindings._rust import asn1_exp


		use crate::asn1_exp::types::{AnnotatedType, AnnotatedTypeObject, Type};

		fn write_value<T: SimpleAsn1Writable>(


		setattr(cls, "__asn1_root__", root)

		def new_init(self: U, /, **kwargs: object) -> None:

Initial implementation of ASN.1 API #13325

Initial implementation of ASN.1 API #13325

Uh oh!

Conversation

facutuesca commented Aug 20, 2025

Overview

Structure

Details

Uh oh!

facutuesca commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alex commented Aug 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

facutuesca commented Aug 20, 2025

Uh oh!

alex commented Aug 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

facutuesca commented Aug 20, 2025 •

edited

Loading

facutuesca Aug 25, 2025 •

edited

Loading