Drop Python3.6 support

[mfeurer]

We currently support more python versions than underlying packages. For us it's 3.6, 3.7, 3.8 and 3.9, while pandas stopped supporting 3.6 a while ago. Also, numpy suggests dropping Python3.6 support in a week. Finally, the latest Ubuntu LTS (20.04) also ships 3.8 by default.

I therefore propose to drop Python 3.6 from the entire Auto-sklearn toolchain to make our lives easier. The affected packages would be the ConfigSpace, SMAC3 and the pynisher. This would then also affect Auto-Pytorch I guess.

CC @KEggensperger @mlindauer @nabenabe0928 @ravinkohli

[ravinkohli]

Hey @mfeurer, thanks for raising this issue. Actually we also very recently discussed removing Python 3.6 support for Auto-PyTorch as it would allow us to use new features from python 3.7+ along with solving the issues with the underlying packages not supporting it.

[mlindauer]

Hi,

If we drop support for some python versions, we could also drop 3.7 in addition, right?

Best,
Marius

[mfeurer]

What would be the benefit of dropping 3.7? So far most issues I've seen arise from 3.6. One downside would be that we loose colab support which currently runs python3.7 by default.

[mlindauer]

The benefit would be that we have to check less Python versions and I would guess it will not take tooooo long before 3.7 support is also dropped. However, colab is very good point -- we should not loose that.

[nabenabe0928]

I agree with Marius and I will add some points.
Note: python3.7 will be officially supported until the end of this year, sooner or later python3.7 will be a not-recommended version officially.

I think the largest benefit from python3.8+ concerning coding parts is typing.
For example, the followings are especially useful to increase the readability of codes:

1. Literal: it can strictly specify what to input

MemoryByteTypes = Literal[32, 64]

def memory_limit(precision: MemoryByteTypes) -> NoReturn:
    ....

2. Final: It can define constant variables

3. TypedDict; It can annotate dict and prevent Dict[str, Any] which does not have almost no information

class Person(TypedDict):
    name: str
    age: int

person0 = Person(name="me", age=25)

# Although it instantiates the class, it is just a dict
>>> person0
{'name': 'me', 'age': 25}

Note: dataclass in python3.7 works similarly. The difference is dataclass handles elements as attributes and TypedDict is just dict after all. According to some sources, there are still some differences in behavior.

4. NewType: It can specify the variable types over whole projects

Name = NewType('Name', str)
Age = NewType('Age', int)

Person = NewType('Person', Tuple[Name, Age])

# Work (guarantees the unity of the name usage)
me = Person((
    Name('me'),
    Age(25)
))

# mypy raises an error
me = Person((
    'me',
    Age(25)
))

[mfeurer]

Thanks @nabenabe0928 for the suggestions, they sound really useful.. We'll stick to >3.7 for now and can drop support for it once Colab moved to Python3.8.