Breaking API changes

[SanderMertens]

A discussion for listing all breaking API changes.

[SanderMertens]

The first branch with breaking changes is about to merge with master. This branch mostly just removes deprecated functions and features, which includes all functions that were marked as deprecated in the last release. Some of the removed functionality (such as bulk APIs) will be reintroduced at a later point, but with a different API design. The highlights are:

• It is no longer possible to add/remove ecs_type_t's to an entity
• Replace legacy filters (include, exclude) with new term-based API
• Remove reader/writer, bulk API, queue, debug API & direct access addons
• Removed all trait-related APIs (use pairs/relations instead)
• Scope iterators have been removed (use term/filter iterators instead)
• ECS_OWNED/flecs::Owned has been replaced with ECS_OVERRIDE/flecs::Override

C++ API

• Remove pod_component, relocatable_component
• Remove entity_range
• Remove action() iterate function
• Add a separate class for triggers (vs. it being lumped together with system)
• Removal of add_childof, add_instanceof (use child_of, is_a)
• children() now accepts a callback function vs. returning an iterator
• The module constructor no longer has the ability to provide a custom name

Some improvements have been made to code organization:

• All C convenience macro's are now in a separate addons/flecs_c.h header
• The C++ API is now a regular addon
• Modules are now regular addons
• All datastructure code has been moved to a separate folder

[SanderMertens]

More things have been updated before the first merge:

• Support for the old query syntax is now entirely removed. See below for a comparison between old/new style
• "superset" and "subset" have been truncated to "super" and "sub" in the query language
• A "parent" modifier has been added which is a shortcut for superset(ChildOf)
• Monitor systems are removed from the core and have been replaced with Observers. See below for the replacement.
• Observers can now be created with a Monitor tag which generate a single event when an entity matches/unmatches
• Filters now no longer match disabled/prefab entities, unless they have a Disabled/Prefab term
• Terms now no longer match disabled/prefab entities, unless explicitly enabled with match_prefab/disabled
• Triggers (and observers) now support terms with superset modifiers
• Triggers (and observers) now support terms with a Not operator
• The .remove field from ecs_entity_desc_t has been removed
• Removed deprecated NOT, OR, XOR type constraints
• The ecs_query_iter function now accepts a world
• Triggers (and observers) can now trigger on custom events, and can specify event wildcards (beta)
• Custom events can be thrown with a new ecs_emit function (beta)

Here are a few examples of the old query syntax vs. the new query syntax:

// old
PARENT:Position
// new
Position(parent)

// old
CASCADE:Position
// new - note the '?', this makes the term optional so that it also matches root entities, which was implicit with CASCADE
?Position(parent|cascade)

// old
Foo:Position
// new
Position(Foo)

// old
:Position
// new
Position()

// old
[out] :*
// new
[out] *()

// old
ANY:Position
// new
Position(self|super)

// old
OWNED:Position
// new
Position(self)

// old
SHARED:Position
// new
Position(super)

// old
OWNED:Likes FOR *
// or
OWNED:TRAIT | Likes > *
// new
Likes(self, *)

Monitor systems are no longer supported and are replaced with monitor observers:

// old
world.system<Position, Velocity>()
  .kind(flecs::Monitor)
  .each([](flecs::iter it) {
    // triggers when entity enters the condition
  });

// new
world.observer<Position, Velocity>()
  .event(flecs::Monitor)
  .each([](flecs::iter it) {
    if (it.event() == flecs::OnAdd) {
      // entity enters condition (matches Position, Velocity for the first time)
    } else {
      // entity exits the condition (no longer matches Position, Velocity)
    }
  });

OnSet systems are no longer supported and are replaced with OnSet observers:

// old
world.system<Position, Velocity>()
  .kind(flecs::OnSet)
  .each([](flecs::iter it) {
    // triggers when entity sets Position or Velocity and has both
  });

// new
world.observer<Position, Velocity>()
  .event(flecs::OnSet)
  .each([](flecs::iter it) {
    // triggers when entity sets Position or Velocity and has both
  });

[logankaser]

Was "// entity exists the condition" supposed to be "// entity exits the condition" or "// entity exists with the condition"?

[SanderMertens]

@logankaser That should've been exits, fixed!

[SanderMertens]

A few breaking changes were introduced for the C++ API:

• checking whether an optional term was set should now be done using the flecs::iter::is_set function (the flecs::column::is_set method was removed)
• Checking if a column is owned should use the flecs::iter::is_owned method (the flecs::column::is_owned / flecs::column::is_shared methods were removed)

[SanderMertens]

Breaking changes were introduced to the C & C++ query APIs:

• The C++ term_builder::subject and term_builder::object methods are renamed to subj and obj
• The C term::args argument has been split into term::subj (previously term::args[0]) and obj (previously term::args[1])
• The world_time member has been removed from ecs_iter_t and flecs::iter. Use ecs_get_world_info instead

// Old (applies to anything that uses `ecs_term_t`, such as `ecs_filter_desc_t` and `ecs_query_desc_t`)
ecs_iter_t it = ecs_filter_init(world, &(ecs_term_t) {
  .args[0].set.mask = EcsSuperSet
});

// New
ecs_iter_t it = ecs_filter_init(world, &(ecs_term_t) {
  .subj.set.mask = EcsSuperSet
});

// Old
auto q = world.query_builder<>()
  .term<Position>().subject().super()
  .build();

// New
auto q = world.query_builder<>()
  .term<Position>().subj().super()
  .build();

[SanderMertens]

The C module API has been revised to reduce module code boilerplate. The module struct and ImportHandles macro are no longer required. The module system is now designed around global component identifiers, which eliminates the need for passing component ids around, and improves interoperability between worlds.

This example shows the difference between the old and new API:

// Old module header
typedef struct {
  float x;
  float y;
} Position;

typedef struct {
  ECS_DECLARE_COMPONENT(Position);
} MyModule;

void MyModuleImport(ecs_world_t *world);

#define MyModuleImportHandles(handles)\
    ECS_IMPORT_COMPONENT(handles, Position);


// Old module source
void MyModuleImport(ecs_world_t *world) {
  ECS_MODULE(world, MyModule);

  ECS_COMPONENT(world, Position);

  ECS_EXPORT_COMPONENT(Position);
}

// New module header
typedef struct {
  float x;
  float y;
} Position;

extern ECS_COMPONENT_DECLARE(Position);

void MyModuleImport(ecs_world_t *world);


// New module source
ECS_COMPONENT_DECLARE(Position);

void MyModuleImport(ecs_world_t *world) {
  ECS_MODULE(world, MyModule);

  ECS_COMPONENT_DEFINE(world, Position);
}

[SanderMertens]

Breaking changes have been introduced to the logging callbacks of the OS API. Additionally the logging API is now an addon.

The log_* functions in the OS API have been replaced with a single function that accepts a logging level, file, line and message. The function no longer accepts a variable argument list.

A number of changes have been made to the logging API that make it more consistent

• ecs_enable_tracing is now called ecs_log_set_level
• ecs_tracing_color_enable is now called ecs_log_enable_colors
• The logging levels have changed to differentiate between tracing and debugging:
  • level >0 = debug
  • level 0 = trace
  • level -2 = warning
  • level -3 = error
  • level -4 = fatal

Level -1 is unused on purpose, as this leaves open the possibility of returning a log level from a function, while still using -1 as the default "this operation has failed" return code.

The following changes have been made to the API:

• There is now a single ecs_log function that accepts a level
• The API has macro's for debug, tracing, warning, error and fatal that call ecs_log
• The API is now an addon. Disabling the addon reduces footprint but also removes tracing & most error reporting
• When the logging addon is disabled, asserts & aborts still work

The messages passed to the OS API have changed:

• They no longer include the log level (this is available as a parameter to the log function)
• They no longer include the indentation (indentation value is stored on the logging API struct)
• They no longer include file and line number (these are available as parameters passed to the function)

[SanderMertens]

Queries with a case term now need to provide both the switch and the case in a term:

// Old
ecs_query_new(world, "CASE | Walking");
// New
ecs_query_new(world, "CASE | (Movement, Walking)");

// Old
ecs_query_init(world, &(ecs_query_desc_t) {
  .filter.terms = {{ .id = ECS_CASE | Walking }}
});
// New
ecs_query_init(world, &(ecs_query_desc_t) {
  .filter.terms = {{ .id = ecs_case(Movement, Walking) }}
});

// Old
world.query_builder<>()
  .term<Movement::Walking>().role(flecs::Case);
// New
world.query_builder<>()
  .term<Movement, Movement::Walking>().role(flecs::Case);

This change was introduced to support creating triggers/observers for terms that match a case, as they need to register themselves for the corresponding switch, which before this change was not known.

The change also brings the switch/case feature closer to pairs. Both features will be merged in the upcoming storage redesign.

[SanderMertens]

The flecs::prefab type no longer exists. Prefabs are now stored as regular entities (with the flecs::entity type). Prefabs can still be created with world.prefab(), which returns a flecs::entity with the Prefab tag.

[SanderMertens]

The C++ API to associate entities/types with a C++ type has changed:

struct Foo { };

// Old
world.entity().component<Foo>();
world.prefab().component<Foo>();
world.type().component<Foo>();

// New
world.entity<Foo>();
world.prefab<Foo>();
world.type<Foo>();

[SanderMertens]

The ecs_query_page_iter and ecs_query_next_worker features have been replaced by chained page/worker iterators that can be used with any source iterator.

Page iterator:

// Old
ecs_iter_t it = ecs_query_page_iter(world, q, 10, 20); // offset 10, limit 20
while (ecs_page_next(&pit)) {
  // iterate as usual
}

// New
ecs_iter_t it = ecs_query_iter(world, q);
ecs_iter_t pit = ecs_page_iter(&it, 10, 20); // offset 10, limit 20
while (ecs_page_next(&pit)) {
  // iterate as usual
}

Worker iterator:

// Old
ecs_iter_t it = ecs_query_iter(world, q);
while (ecs_query_next_worker(&pit, 0, 2)) {  // worker id 0, worker count 2
  // iterate as usual
}

// New
ecs_iter_t it = ecs_query_iter(world, q);
ecs_iter_t wit = ecs_worker_iter(&it, 0, 2); // worker id 0, worker count 2
while (ecs_worker_next(&wit)) {
  // iterate as usual
}

[SanderMertens]

When parsing query identifiers, single upper-case letters are no longer treated as variables. All variables must now be prefixed with _.

// Old
ecs_rule_t *r = ecs_rule_init(world, &(ecs_filter_desc_t) {
    .expr = "Likes(X, Y)"
});

int32_t x = ecs_rule_find_variable(r, "X");
int32_t y = ecs_rule_find_variable(r, "Y");

// New
ecs_rule_t *r = ecs_rule_init(world, &(ecs_filter_desc_t) {
    .expr = "Likes(_X, _Y)"
});

int32_t x = ecs_rule_find_variable(r, "X");
int32_t y = ecs_rule_find_variable(r, "Y");

[SanderMertens]

The on_set callback signature (as set in the EcsComponentLifecycle struct) has been changed to match that of iterators (void(*)(ecs_iter_t *).

[SanderMertens]

The ecs_type_index_of and ecs_type_has_id functions have been replaced by ecs_search, ecs_search_offset and ecs_search_relation.

[SanderMertens]

A few methods of the flecs::iter class have been renamed:

• flecs::iter::term_id is renamed to flecs::iter::id
• flecs::iter::term_source is renamed to flecs::iter::source

Additionally, tag arguments (empty types) can no longer be passed as references to each, which addresses a UB issue. The following code will now no longer compile:

struct Tag { };

auto q = world.query<Tag>();

q.each([](flecs::entity e, Tag&) { }); // Illegal, cannot be a reference

The correct way to add the tag argument is as a regular value:

struct Tag { };

auto q = world.query<Tag>();

q.each([](flecs::entity e, Tag) { }); // Ok

Passing a tag as argument doesn't have much added value though, so another alternative is to do this:

struct Tag { };

auto q = world.query_builder().term<Tag>().build();

q.each([](flecs::entity e) { }); // Ok

[SanderMertens]

The C++ flecs::map wrapper has been removed. The reason is that it is a datastructure optimized for the usage in the library, and does not have features for things general purpose should contain, such as resource management.

[SanderMertens]

Specifying a component using the query builder with .with now uses the InOutDefault inout kind (previously InOutNone). For most scenarios this should not cause existing code to break, however this scenario will now throw an assert:

auto q = world.query_builder()
  .with<Position>().src(e)
  .build();

q.run([](flecs::iter& it) {
  while (it.next()) {
    auto p = it.field<Position>(0); // access violation: component is readonly
  }
});

This happens because with InOutDefault, terms with a non-$this source automatically become readonly. To fix the assert either add .inout() to the builder:

auto q = world.query_builder()
  .with<Position>().src(e).inout()
  .build();

or add const to the code fetching the field:

q.run([](flecs::iter& it) {
  while (it.next()) {
    auto p = it.field<const Position>(0);
  }
});

[SanderMertens]

The flecs::entity::remove method had an issue where if it was used with an enum type, it would remove an enum relationship. This prevented the function from being used to remove enum components. To remove an enum relationship, the function now can be called with a Wildcard argument:

// Old
enum Color {
  Red, Green, Blue
};

e.add(Color::Red); // adds (Color, Red)
e.remove<Color>(); // removes (Color, *)

e.set<Color>(Red); // adds Color { Red }
e.remove<Color>(); // doesn't remove Color

// New
enum Color {
  Red, Green, Blue
};

e.add(Color::Red); // adds (Color, Red)
e.remove<Color>(flecs::Wildcard) // removes (Color, *)

e.set<Color>(Red); // adds Color { Red }
e.remove<Color>(); // removes Color

[SanderMertens]

Change detection now has to be explicitly enabled on the queries that are used to detect changes. When change detection is not enabled, calling ecs_query_changed or ecs_iter_changed will fail.

// C
ecs_query_t *q = ecs_query(world, {
  .terms = { ... },
  .flags = EcsQueryDetectChanges
});

// C++
flecs::query<...> q = world.query_builder<...>()
  .detect_changes()
  .build();

[SanderMertens]

It is no longer valid to pass 0 to the size argument of ecs_field_w_size. A valid size must now be passed into the function, or the code will assert. If the size is not known at compile time, code can use ecs_field_size:

// Old
void *ptr = ecs_field_w_size(&it, 0, 1);

// New
void *ptr = ecs_field_w_size(&it, ecs_field_size(&it, 0), 1);

[SanderMertens]

It is no longer allowed to create new entities from a multithreaded system. The reason for dropping support are:

• Previous performance improvements already prevented applications from using entity ids created in a multithreaded system, since the entity wasn't alive yet. This means id generation from multithreaded systems wasn't very useful.
• The multithreaded code added a performance overhead to all entity creation
• Multithreaded entity creation didn't use recycling, which meant that doing this could leak ids.

[SanderMertens]

The entity_view::get and entity_view::get_mut methods now return a reference instead of a pointer:

// Old
const Position *p = e.get<Position>();
Position *p_mut = e.get_mut<Position>();

// New
const Position& p = e.get<Position>();
Position& p_mut = e.get_mut<Position>();

If the entity does not have the component, get and try_get will panic. New try_get and try_get_mut methods have been added that implement the old behavior:

const Position *p = e.try_get<Position>();
Position *p_mut = e.try_get_mut<Position>();

[SanderMertens]

The entity::get method no longer returns the enumeration constant value of an enum relationship when used with an enumeration type. A new get_constant method has been added that provides similar behavior:

// add enum relationship pair
e.add(Color::Green);

// Old
Color *c = e.get<Color>();

// New
Color c = e.get_constant<Color>();

[SanderMertens]

Union relationships are no longer supported, and have been replaced with DontFragment, Exclusive relationships. DontFragment, Exclusive relationships are functionally almost entirely equivalent to Union, and come with a few additional advantages such as the ability to attach data to a relationship.

Most code should still work as expected after replacing the Union trait with the DontFragment/Exclusive traits:

C:

// Old
ecs_add_id(world, ecs_id(Position), EcsUnion);

// New
ecs_add_id(world, ecs_id(Position), EcsDontFragment);
ecs_add_id(world, ecs_id(Position), EcsExclusive);

C++:

// Old
world.component<Position>().add(flecs::Union);

// New
world.component<Position>().add(flecs::DontFragment);
world.component<Position>().add(flecs::Exclusive);

The most visible difference from the API perspective is that whereas previously entities with a union relationship would have a (R, Union) pair in their table/type, this no longer happens with DontFragment relationships. Adding or removing a DontFragment relationship will never change the table/type of the entity.

There are some limitations when compared with normal relationships. See the documentation for an up to date list.

[SanderMertens]

The ecs_emplace_id and ecs_ensure_id functions now accept an additional size argument:

// Old
void *ptr = ecs_ensure_id(world, entity, component_id);

// New
void *ptr = ecs_ensure_id(world, entity, component_id, component_size);

// Old
bool is_new = false;
void *ptr = ecs_emplace_id(world, entity, component_id, &is_new);

// New
bool is_new = false;
void *ptr = ecs_emplace_id(world, entity, component_id, component_size, &is_new);

If the size argument does not match the size of the component, the functions may panic.

[SanderMertens]

The singleton API has changed to a component trait, and no longer requires queries to be annotated with .singleton:

C:

// Old
ECS_COMPONENT(world, TimeOfDay);

ecs_singleton_set(world, TimeOfDay, {0});

ecs_query_t *q = ecs_query(world, {
  .terms = {{ ecs_id(TimeOfDay), .src = EcsSingleton }}
});

// New
ECS_COMPONENT(world, TimeOfDay);

ecs_add_id(world, ecs_id(TimeOfDay), EcsSingleton);

ecs_singleton_set(world, TimeOfDay, {0});

ecs_query_t *q = ecs_query(world, {
  .terms = {{ ecs_id(TimeOfDay), .src = EcsSingleton }}
});

C++:

// Old
world.component<TimeOfDay>();

world.set(TimeOfDay{0});

auto q = world.query_builder<TimeOfDay>()
  .term_at(0).singleton()
  .build();

// New
world.component<TimeOfDay>().add(flecs::Singleton);

world.set(TimeOfDay{0});

auto q = world.query<TimeOfDay>();

Query DSL:

// Old
TimeOfDay($)

// New
TimeOfDay

The singleton trait will enforce that the component can only be added to itself. Attempting to add the component to a different entity will panic.

[SanderMertens]

The ecs_script_parse, ecs_script_eval and ecs_script_run functions now have an additional result argument which allows for capturing the error of script execution:

ecs_script_eval_result result = {0};
if (ecs_script_run(world, "my script", "e {", &result)) {
  printf("running script failed: %s\n", result.error);
  ecs_os_free(result.error);
}

[SanderMertens]

Inheritance statements in Flecs script now need to end with a scope:

// This is no longer allowed
entity : Base

// Now always need to add a scope (just like for regular entity statements)
entity : Base { }

This change helps to prevent ambiguities with the new new expressions.

[SanderMertens]

The ecs_script_init() function no longer returns 0 when a script fails to load. ecs_script_init() (as opposed to ecs_script_run(), ecs_script_parse(), ecs_script_eval()) creates a "managed script", which is a script that a user can update with new code which will automatically reload.

To allow for scenarios where the user is live editing the code, the script object needs to persist even when the code contains errors. This was possible with the ecs_script_update() function, but because ecs_script_init() returned 0 when the script contains errors, it was not possible to create a managed script that contains errors.

This could cause issues, where an application would attempt to load a managed script with errors, which then failed, which would prevent a user from fixing errors in tools like the explorer while the application is running.

This change fixes that workflow, and makes sure the behavior of ecs_script_init() is consistent with updating scripts after script creation.

[SanderMertens]

The meta (reflection) API has changed to fix a number of issues related to serializing and deserializing (opaque) vectors. The following things changed:

• The count member for EcsMember / ecs_member_t must be left to 0 for normal members. Any non-zero number will be interpreted as an inline array (previously any number >1 would be treated as inline array)
• The constants member in the Enum and Bitmask components is now a pointer to a map instead of an inlined map value. This allows for faster serializing of enums/bitmasks since the map can be cached by the serializer.

The following changes have been made to serializer instructions (see the new custom_serializer examples for how to use them):

• The EcsOpPush constant has been renamed to EcsOpPushStruct
• An EcsOpPushArray and EcsOpPushVector instruction have been added. These are added as instruction to array/vector types, which now makes handling collections very similar to handling structs (start with Push, end with Pop)
• The EcsOpOpaque instruction has been split up into EcsOpOpaqueStruct, EcsOpOpaqueArray, EcsOpOpaqueVector and EcsOpOpaqueValue
• A new EcsOpForward instruction has been added that forwards to another type
• ecs_meta_type_op_t has been renamed to ecs_meta_op_t
• ecs_meta_type_op_t::members has been changed to a union that can either contain members, constants or a callback to an opaque type serializer

C serializer example: https://github.com/SanderMertens/flecs/blob/master/examples/c/reflection/custom_serializer/src/main.c

C++ serializer example:
https://github.com/SanderMertens/flecs/blob/master/examples/cpp/reflection/custom_serializer/src/main.cpp

[SanderMertens]

The constants member of EcsEnum and EcsBitmask has been moved to a new EcsConstants component. The reason behind this is that the constants member was automatically populated, and applications could accidentally overwrite this value when assigning the enum component:

// resets .constants to nullptr
e.set(Enum{ .underlying_type = some_type });