metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

[dghubble]

Some time in the last 12 hours, our build system using gsutil started failing and it appears to be related to six 1.11.0, which was just released.

$ gsutil
...
 File "myproject/venv/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py", line 1168, in <module>
    class Field(six.with_metaclass(_FieldMeta, object)):
TypeError: Error when calling the metaclass bases
    metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

gsutil declares a dependency on six>=1.9.0 (ugh) so it pulls in the just released 1.11.0. Rolling back to six 1.10.0 and gsutil runs alright. I'm not really involved in Python or gsutil, so maybe gsutil needs to fix something, no idea, but it seems like a breakage that's not called out in the release notes or maybe something to fix.

cc @jdemeyer @benjaminp
Possibly introduced in #191

[cpoppema]

Just stumbled onto this as well. Not using gsutil. Another package (django-extensions) has six>=1.2 as a dependency. For now, adding 1.10.0 to the requirements myself as a temporary workaround.

[jdemeyer]

A minimal non-working example would be nice.

[cpoppema]

Looking at the 3rd party package code that triggers the error, it boils down to this:

import six


class Meta(type):
    pass


class A(six.with_metaclass(Meta)):
    __metaclass__ = Meta

Removing the __metaclass__ prevents the TypeError.

[jdemeyer]

Removing the metaclass prevents the TypeError.

Obviously, that is the right fix. The whole point of using six.with_metaclass is that you do not need __metaclass__ (which is not supported in Python 3).

That being said, if this broke actual usage (even if that actual usage is arguably wrong), it makes sense to try to fix this.

[jdemeyer]

IOError: [Errno 2] No such file or directory: '/opt/test/lib/python2.7/site-packages/six-1.10.0.dist-info/METADATA'

This is obviously unrelated to this issue, it seems more like an installation issue.

[cpoppema]

I'll create a pull request to remove the __metaclass__ from my example. Not sure if @dghubble can spot a similar pattern or if it's caused by something different.

[jdemeyer]

I could probably manage to fix the

class A(six.with_metaclass(Meta)):
    __metaclass__ = Meta

example but it will lead to more complicated code for six.with_metaclass.

I would like some input from the six maintainers on how to proceed.

[florenthemmi]

I have the same issue with some Google libraries:

  File "/Users/florenthemmi/dev/env/lib/python2.7/site-packages/apache_beam/internal/gcp/json_value.py", line 23, in <module>
    from apitools.base.py import extra_types
  File "/Users/florenthemmi/dev/env/lib/python2.7/site-packages/apitools/base/py/__init__.py", line 21, in <module>
    from apitools.base.py.base_api import *
  File "/Users/florenthemmi/dev/env/lib/python2.7/site-packages/apitools/base/py/base_api.py", line 31, in <module>
    from apitools.base.protorpclite import message_types
  File "/Users/florenthemmi/dev/env/lib/python2.7/site-packages/apitools/base/protorpclite/message_types.py", line 25, in <module>
    from apitools.base.protorpclite import messages
  File "/Users/florenthemmi/dev/env/lib/python2.7/site-packages/apitools/base/protorpclite/messages.py", line 1165, in <module>
    class Field(six.with_metaclass(_FieldMeta, object)):
TypeError: Error when calling the metaclass bases
    metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

[jdemeyer]

The issue in google-apitools is the same:

# TODO(rafek): Prevent additional field subclasses.
class Field(six.with_metaclass(_FieldMeta, object)):
    """Definition for message field."""

    __initialized = False  # pylint:disable=invalid-name
    __variant_to_type = {}  # pylint:disable=invalid-name

    # TODO(craigcitro): Remove this alias.
    #
    # We add an alias here for backwards compatibility; note that in
    # python3, this attribute will silently be ignored.
    __metaclass__ = _FieldMeta

[drewjaja]

Got the same issue using a Django package.

File "/usr/local/lib/python2.7/site-packages/autocomplete_light/shortcuts.py", line 8, in <module>
   from .forms import *
 File "/usr/local/lib/python2.7/site-packages/autocomplete_light/forms.py", line 438, in <module>
   class ModelForm(six.with_metaclass(*bases)):
 File "/usr/local/lib/python2.7/site-packages/autocomplete_light/forms.py", line 283, in __new__
   attrs)
 File "/usr/local/lib/python2.7/site-packages/django/forms/models.py", line 208, in __new__
   new_class = super(ModelFormMetaclass, mcs).__new__(mcs, name, bases, attrs)
 File "/usr/local/lib/python2.7/site-packages/django/forms/forms.py", line 41, in __new__
   new_class = super(DeclarativeFieldsMetaclass, mcs).__new__(mcs, name, bases, attrs)
 File "/usr/local/lib/python2.7/site-packages/django/forms/widgets.py", line 153, in __new__
   new_class = super(MediaDefiningClass, mcs).__new__(mcs, name, bases, attrs)
TypeError: Error when calling the metaclass bases
   metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

[jdemeyer]

Which version of django-autocomplete-light is that?

[drewjaja]

django-autocomplete-light 2.3.3

[jdemeyer]

Same issue with django-autocomplete-light 2.3.3:

class ModelForm(six.with_metaclass(*bases)):
    """
    ModelForm override using our metaclass that adds our various mixins.

    .. py:attribute:: autocomplete_fields

        A list of field names on which you want automatic autocomplete fields.

    .. py:attribute:: autocomplete_exclude

        A list of field names on which you do not want automatic autocomplete
        fields.

    .. py:attribute:: autocomplete_names

        A dict of ``field_name: AutocompleteName`` to override the default
        autocomplete that would be used for a field.

    Note: all of ``autocomplete_fields``, ``autocomplete_exclude`` and
    ``autocomplete_names`` understand generic foreign key and generic many to
    many descriptor names.
    """
    __metaclass__ = ModelFormMetaclass

[ivanpricewaycom]

same for me using the module boxsdk.

just need to:
pip install six==1.11.0 boxsdk==2.0.0a8
python -c "import boxsdk"

et bim:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/boxsdk/__init__.py", line 6, in <module>
    from .client import *  # pylint:disable=wildcard-import,redefined-builtin
  File "/usr/local/lib/python2.7/site-packages/boxsdk/client/__init__.py", line 5, in <module>
    from .client import Client
  File "/usr/local/lib/python2.7/site-packages/boxsdk/client/client.py", line 12, in <module>
    from ..object.events import Events
  File "/usr/local/lib/python2.7/site-packages/boxsdk/object/events.py", line 15, in <module>
    class EventsStreamType(with_metaclass(ExtendableEnumMeta, TextEnum)):
  File "/usr/local/lib/python2.7/site-packages/boxsdk/util/compat.py", line 77, in with_metaclass
    return type.__new__(TemporaryMetaSubclass, str('temporary_class'), bases, {})
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

using six 1.10.0 there is no problem.

[jdemeyer]

boxsdk is more interesting because it uses six.with_metaclass in an unusual way to define a variant of with_metaclass:

def with_metaclass(meta, *bases, **with_metaclass_kwargs):
    """Extends the behavior of six.with_metaclass.
    [...]
    """
    temporary_class = six.with_metaclass(meta, *bases, **with_metaclass_kwargs)
    temporary_metaclass = type(temporary_class)

    class TemporaryMetaSubclass(temporary_metaclass):
        @classmethod
        def __prepare__(cls, name, this_bases, **kwds):  # pylint:disable=unused-argument
            return meta.__prepare__(name, bases, **kwds)

    return type.__new__(TemporaryMetaSubclass, str('temporary_class'), bases, {})

This is using with_metaclass internals, so it shouldn't be a surprise that it breaks.

It would be better if boxsdk would have a complete self-contained implementation if they don't like six.with_metaclass. The current mixture of six.with_metaclass with custom code is what causes the trouble.