bug(python): Potential breaking change in protobuf 5.27.0rc1 where `MessageMapContainer` cannot be imported

[parthea]

What version of protobuf and what language are you using?
Version: main/v3.6.0/v3.5.0 etc. (NOTE: please try updating to the latest version of protoc/runtime possible beforehand to attempt to resolve your problem)
Language: C++/Java/Python/C#/Ruby/PHP/Objective-C/Javascript
Python Protobuf 5.27.0rc1

What operating system (Linux, Windows, ...) and version?
Debian GNU/Linux rodete
6.6.15-2rodete2-amd64

What runtime / compiler are you using (e.g., python version or gcc version)
Python 3.9.16
gcc version 13.2.0 (Debian 13.2.0-10)

What did you do?
Steps to reproduce the behavior:

1. Create foo.proto

syntax = "proto3";

message Foo {
  int32 bar = 1;
}

message Baz {
    map<string, Foo> foos = 1;
}

2. Compile proto using protoc --python_out=. foo.proto
3. Run the following python code and observe that the type is <class 'google._upb._message.MessageMapContainer'>

import foo_pb2
foo = foo_pb2.Foo(bar=5)
type(foo_pb2.Baz(foos = {"i": foo}).foos)

>>> import foo_pb2
>>> foo = foo_pb2.Foo(bar=5)
>>> type(foo_pb2.Baz(foos = {"i": foo}).foos)
<class 'google._upb._message.MessageMapContainer'>

5. Run the code below which results in ImportError
   from google._upb._message import MessageMapContainer

What did you expect to see
I expected from google._upb._message import MessageMapContainer to succeed as it works in 5.26.1

(py392) partheniou@partheniou-vm-3:$ python3
Python 3.9.16 (main, Mar  2 2023, 17:52:22) 
[GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google._upb._message import MessageMapContainer
>>> import google.protobuf
>>> google.protobuf.__version__
'5.26.1'

(py392) partheniou@partheniou-vm-3:~$ python3
Python 3.9.16 (main, Mar  2 2023, 17:52:22) 
[GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from google._upb._message import MessageMapContainer
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'MessageMapContainer' from 'google._upb._message' (/usr/local/google/home/partheniou/.pyenv/versions/3.9.16/envs/py392/lib/python3.9/site-packages/google/_upb/_message.abi3.so)
>>> import google.protobuf
>>> google.protobuf.__version__
'5.27.0rc1'

What did you see instead?

>>> from google._upb._message import MessageMapContainer
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'MessageMapContainer' from 'google._upb._message' (/usr/local/google/home/partheniou/.pyenv/versions/3.9.16/envs/py392/lib/python3.9/site-packages/google/_upb/_message.abi3.so)

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment

[anandolee]

Chatted with Anthonios, and this is the update:

#15999 might the PR that introduced the change

from google._upb._message import MessageMapContainer which import internal type is not a use case we want to support. The reason to import the container is mostly check if a field is repeated, the right way is use field descriptor instead like:

if field_des.label == descriptor.FieldDescriptor.LABEL_REPEATED:
    if field_des.message_type:
        // It is a repeated message
    else:
        // It is a repeated scalar

[brcsw]

The latest version of the gcloud CLI seems to be broken due to this issue: https://issuetracker.google.com/issues/343748810

[parthea]

Closing as obsolete. This was part of an internal protobuf API rather than the public interface.