docs: extending/adding backends

[achapkowski]

Hello,

Let's say I want to add a new database backend, is there an existing implementation I should follow to extend it?

I looked at the doc: http://ibis-project.org/docs/user_guide/sql.html but it is a bit lacking.

Thank you

[datapythonista]

Thanks for the interest @achapkowski. We don't have much documentation at the moment unfortunately. This is the relevant section, but there is not much there.

You can check these pages that can help a bit:

• To understand the classes of backends: https://ibis-project.org/docs/backends/index.html#classes-of-backends
• Some internal documentation about the OmniSci backend: https://ibis-project.org/docs/backends/omnisci.html#backend-internals

In short, I'd say that the best would be to have a SQLAlchemy backend for the database you are interested on. You can see the available dialects here: https://docs.sqlalchemy.org/en/13/dialects/

Then, the main reference I would use is the MS SQL server backend. The main reason is that it's a third-party backend, so it'll be easier to understand what's the backend, and what Ibis itself. But also, it's the newest, so I guess the code should be more up to date.

You're very welcome to help improve the documentation. I'm working on it, but there is a lot of work to make it as useful as it should be.

@costrouc, @xmnlab is there anything else you want to add to my comments?

[xmnlab]

@achapkowski the information @datapythonista is a good start point.

I think the third-party approach used by @costrouc is very good because you can isolate your backend CI infrastructure.
As currently, there are no tests exclusive for a third-party approach, probably you would need to create your own tests. it would be good to have a separated repo just for tests that could be reused for any third-party backend, but there is nothing available for that at this moment.

[saulpw]

Needed for this:

• toplevel architectural prose
• well-known index page for backend developers
• explanation of minimal backend (sqlite as reference?)
• flesh out API docs for backend classes and methods that need to be implemented