Hello everyone, I am back again and so is (finally!) a new official Svelto.ECS update! It sounds almost incredible, but Svelto.ECS 3.0 is the result of exactly one year of work! I can’t believe it. Initially, I planned to take at most three months to release the features I had in mind, but then I decided to make Svelto Unity Jobs and Burst compatible and there it went most of the unplanned time.

Svelto.ECS 3.0 has been used for the last 18 months on the new Freejam’s product Gamecraft. Gamecraft is full of technical challenges, including the necessity to physic simulate thousands of entities. Gamecraft is also the first product I developed to be 100% ECS (wrapping OOP code where necessary).

MAJOR UPDATE: Destruction, Quantum Rifle More - NOW LIVE · Gamecraft update  for 21 August 2020 · SteamDB
Gamecraft has some intensive features

I hope to write new articles in the near future to explain the most important things I learned during the development of Gamecraft, but this article is going just to be a brief Svelto.ECS 3.0 introduction as the jump from Svelto.ECS 2.9 is quite big and there is a LOT to talk about. As you can imagine old code would need some rethinking to work with the new patterns/features. For this reason, is probably better for me to start from scratch so that we can refresh some terminology and ideas.

Before to continue, remember that anyone can contact me and other Svelto users joining the official discord channel which is the best way to get help. The Svelto.Miniexamples and Svelto.ECS.Tests repositories are the best places where to find ready to study and up to date examples.

  • This article is not an introduction to ECS neither is intended to explain how to use Svelto.ECS as some concepts are too advanced to be crammed in one article only. MiniExamples and Svelto Discord server are actually the best places where to start.

The Basics: EnginesRoot and CompositionRoot

The most important class remains the so called EnginesRoot. This is the class that holds all the data and systems to run your logic. You will find it in all the MiniExamples examples. If you know Unity ECS, it’s similar to the World concept. Similarly to it, you can have multiple EnginesRoots to encapsulate even further data and logic.

Contrary to UECS, Svelto tends to not hide complexity and be as simple as possible. Svelto does not create default EnginesRoots or systems automatically, so everything must be instantiated and injected explicitly through a so-called Composition Root. Refer to my previous articles to know what a Composition Root is if necessary, but essentially it’s the bootstrap code where the Enginesroot, the Scheduler and the Engines are created. Remember that in Svelto.ECS I still refer to the ECS systems as engines (even because I am not 100% sure that System is the correct word to use, but this is another story).

A composition root can take many forms as it’s not how it’s designed that it matters, but what it does. An example can be:

Note that there are no rules on what you can inject in an engine, so the Composition Root is also where all the dependencies are resolved.


The Basics: Entity Descriptors and Entity Components

Svelto.ECS entities are described by EntityDescriptors. The concept may remind the Archetype in UECS. Entity descriptors define the list of Entity Components (previously called Entity Structs) that each entity generates. However, Entity descriptors are more important than that. They were initially designed to never let the user forget what an entity is. Using Svelto, the user will not fall in the trap to reason only in terms of components. Having always in mind what an entity is is fundamental as engines should be designed around entities and not decontextualised components. One rule I always push for is to name the engines after the Behaviour to apply + Name of the entity the engine operates on so that these concepts are visualised in code.

Entity Components in Svelto are always structs and must always hold value types only. The simplest way to declare an entity descriptor looks like:

it’s important to note that SECS has been designed to push the user mentality away from the classic definition of an entity being just an ID. SECS instead promotes the conceptual aspect of an entity. Personally, I like to have the rule that an entity must reflect a design entity that the user can experience. In doing so, the name of the entity usually comes from the Game Design Document. This also promotes ubiquitous language between designers and coders.


The Basics: The EntitySubmissionScheduler

Since the first time Svelto has been used in production, created entities have been submitted at once during the Submission time. One fundamental feature of Svelto is that it doesn’t provide any engine ticking solution. The user has the responsibility to decide when to run engines and when entities are submitted. Two predefined EntitySubmissionScheduler are provided:

UnityEntitiesSubmissionScheduler, as seen in MiniExample 2: Survival. It’s a Unity dedicated submission scheduler that submits new entities every unity update. This is used for simple demos and probably would never be used in anything more complicated than that.

SimpleEntitiesSubmissionScheduler, as seen in MiniExample 4: SDL. It’s the suggested way to submit entities. The user has full control on when the entities should be submitted.

During the submission time, all the Add, Remove and MoveTo callbacks are executed, reacting on entities added, removed or swapped (swap is a concept linked to groups, as you will see in a bit). I refer to these operations as structural changes.


The Basics: EntityViewComponents, Implementors and Hybrid ECS

Although Svelto is designed to allow 100% ECS based applications, not all the platforms/libraries are written with ECS based code. Hence it’s still necessary to work side by side with OOP entities. Mixing ECS code with OOP is and always be awkward, consequentially there are multiple ways to approach this problem. Svelto provides one officially, others are patterns that will be explained in future articles. The official way is the use of EntityViewComponents, that are entity components that can hold interfaces that are implemented with objects. These objects are called implementors in Svelto. While the object is a standard c# object, the interface has limited flexibility, for example, it can expose only properties that work on value types. With EntityViewComponents performance is reduced, as the memory layout won’t be cache-friendly anymore. They are called like this because EntityViewComponents were initially designed to wrap the objects that should act as view. I like to draw a parallel with the MVP pattern. The presenter is the engine, the model is the data that can be found in entity component, the view is the implementor that acts on data change. While the MVP pattern works vertically (a triad for each element), in ECS this works horizontally (the engine operates on all the views at the same time). A sort of data binding mechanism is also present. Some of these features are shown in MiniExample 3: GUI. EntityViewComponents are also used in MiniExample2: Survival to wrap Gameobjects.


The Basics: ExclusiveGroups

Every ECS framework implementation has its own strategy to store entities most efficiently. With SECS, I designed a novel solution that is based on the use of Groups. I see the groups as a way to decouple the state of the entity from its values. In general, the user would find it safe to create a group for each entity descriptor. This would act similarly to how Archetypes work in UECS (except for the further split in different chunks). An entity belongs to a group at the time (hence the name ExclusiveGroup) and the user can freely swap entities between groups when necessary. When entities are built, removed, swapped between groups and iterated, the user must declare which group the operation must act on. Groups are used in all the MiniExamples and while they are initially not intuitive, the user usually gets them quickly. In any case, ExclusiveGroups should never be created at runtime (this still needs to be enforced by the framework) and are always declared as static variables.

Groups can be seen as the way to organise the entity queries. The user queries entities according to the groups where they must be taken from. In this way is possible, for example, to iterate entities in relation to their states or meaning. SECS allows queries from single groups or array of groups. Arrays of groups are usually associated with abstract engines or engines that iterate entities sharing common states. For example, in MiniExample 2: Survival:

the DeathEngine engine shows:

The DeathEngine is more abstract than Player or Enemies entities related engines because it iterates both entities types assuming that HealthComponent and DeathComponent will be present in the respective descriptors.

Rearranging the meaning of the entities through the use of Groups opens the door to interesting strategies, however handling ExclusiveGroups explicitly can get awkward fast, that’s why GroupCompounds have been introduced with Svelto.ECS 3.0.

ExclusiveGroups aren’t completely replaced by the newly introduced GroupCompounds because of possible advanced scenarios. One feature that is still related only to ExclusiveGroups is the option to reserve a range of groups for procedural queries. ExclusiveGroup declaration can look like:

Reserved ExclusiveGroups can then be used with the name of the group + index. In this case, for example, it can be used as PopupInputItemsGroup + N where N is between 1 and 99. Being an advanced scenario, don’t worry if you don’t understand why to use reserved groups while reading this article, but knows that MiniExamples 2: Survival uses them.


New Feature: GroupCompounds

GroupCompound should be the default way to declare groups in Svelto and the new user should start to use them in place of the conventional ExclusiveGroups. GroupCompounds are designed to statically manage groups based on tags, making the use of groups simpler, more intuitive and more powerful. This is an example of the use of group compounds:

The code above is a SECS group declaration pattern and although classes are used they will never be instantiated and can in fact be declared as abstract for further peace of mind.

One unique feature of Svelto.ECS is to prohibit adding or removing entity components at execution time (avoiding the so-called component tagging practice). Once an entity descriptor (and therefore the entity) is defined, it’s immutable. This feature has several performance benefits, but it is mainly designed to prevent distorting the definition of an entity. If an entity can become anything at any time, the distinctive entity declaration would become meaningless (and the user would revert thinking in terms of decontextualised components instead of entities).

GroupCompounds is one of the two new features that come in to play to fill the lack of dynamic component tagging which was limiting the development of complex features with Svelto.ECS 2.x. When looking at groups as “compound” of tags, is possible to switch states or adjectives just swapping an entity from a compound to another compound.

An entity that is found in the group DynamicRigidBodyWithBoxColliders could move to the group KinematicRigidBodyWithBoxColliders allowing different engines to run on the entity just swapped. This, together with the use of the MoveTo callback, could enable the development of simple state machines.

GroupCompounds are usually designed with the first tag being the entity type and the successive tags being states or adjectives of the entity instance.

GroupCompounds are used in the examples MiniExamples 1: Doofuses and MiniExamples 4: SDL.


Pattern: how to build entities

Putting all together, the syntax to build an entity looks like:

An entity is built using its descriptor. Then, and this is a unique Svelto feature, the user needs to associate an ID to the entity and at last, the user needs also to choose in which group the entity must be put in. The combination of Entity ID and Group ID in Svelto is called EGID. Because the entity group can change after a swap, an EGID value is not constant. In the next versions of Svelto.ECS, I will introduce the already planned EntityReference feature, which will abstract the EGID and will enable simpler ways to link entities to other entities.

the EGID can be successively used to query specific entities. Linking a user-defined ID to entities enables more patterns compared to other ECS framework implementations. The returned initializer is a transient container that allows initialising the components created if needed.

Remember that once an entity is built it’s not immediately available. It will be available only after the next submission of entities.


Pattern: how to iterate entities

There are several ways to iterate entities, but with the use of group compounds the user will end up resorting to similar patterns, looking like:

The use of deconstructing patterns is elegant but weird at first. The user will get used to it soon and I suggest to use your IDE ability to work out the deconstruction automatically at first.

you could start using a random variable name (the IDE (in this case Rider) knows is not used and it will grey it out)
then ask the IDE to deconstruct the iterating item
and eventually you can rename the parameters

Multiple components can be queried and used at once inside the iteration (using the same index). This is really as fast as an iteration can be and highly suitable for vectorisation.

In the following example:

the user is asking to get the RigidbodyEntityComponent and TransformEntityComponent components for each entity for each group where DynamicRigidBodies entities can be found. The foreach will iterate over all the arrays found in the separate groups, regardless how many they are. rigidbodies and transforms are obviously arrays. count is their length. group is the current group iterated. The user then can iterate the components knowing they belong to the same entity for each index used in the arrays iteration.

Several examples of entities iteration can be found in the MiniExamples repository.


New Feature: Filters

Like GroupCompounds, the new Filters feature allow entities to be iterated according to their states. GroupCompounds are powerful, but also relatively rigid. For once entities can be swapped only during the submission time. Although the iteration of entities through group compounds is as fast as it can be, group compounds split groups in memory. This is why the number of compound tags is limited to four so that states permutations won’t lead to an excessive segmentation of the memory.

Filters, on the other hand, allow the creation of subgroups through indexing. Filters are nothing more complex than an array of indices of subsets of entities found in given groups. GroupCompounds and Filters are orthogonal. GroupCompounds divide entities into multiple arrays, which benefits specialised engines, but penalised abstract engines. Filters instead usually work on simpler ExclusiveGroups that are not split further than the EntityDescriptor group. Filters allow faster abstract engines (since there is no need to jump between groups) and slower specialised engines due to the double indexing needed for the filtering. Unluckily I don’t have a mini example using filters yet, so I will need to fill this gap soon. The use of GroupCompounds and Filters together is possible, but it has never been tested extensively so far (meaning that precise patterns haven’t been identified yet).

An example of Filters usage is:

the example above shows that multiple filters can exist for each component for each group and they are identified with an ID. The new filters feature needs a separate article which will come at a given point. Meanwhile the best option for you is to ask about them on the Svelto discord channel if needed.


Review: How groups work

Using GroupCompounds and specifying an ExclusiveGroup for each entity descriptor are the foolproof strategies to never experience problems with SECS queries. More advanced usages of groups are for users who truly understand SECS database memory layout.

Fundamentally the memory layout is simple: An array for each component type is used to store the components generated by each entity. The use of simple arrays and components as value types guarantee the components iteration to be cache-friendy and vectorizable.

However, it’s common for the user to want to iterate multiple components expecting that each tuple of components belongs to the same entity. Fulfilling this expectation is what makes each ECS library implementation different. SECS solves this problem explicitly with the use of groups. As long as ALL the entities present in a group to query generate the tuple of components to iterate, SECS guarantees that each tuple iterated belongs to the same entity.

This problem is simpler to visualise than to explain:

An entity descriptor can generate several components. Let’s say that E1 are entities generated from EntityDescriptor1 and E2 are entities generated from EntityDescriptor2. Let’s say that E1 generates EntityComponentA (E1-CA) and E2 generates EntityComponentA (E2-CA) and EntityComponentB (E2-CB). This means that E1 and E2 have components of type A in common, while only E2 have components of type B. If the user builds five E1 entities and three E2 entities using the same group, in memory the components are laid like this:

Once an engine queries for the components CA and CB using the group specified during the building, the CA and CB components arrays are returned exactly as shown in the example above. Using the same index will now result in mixing components from different entities. In any case, SECS won’t even allow this query, as, at least, the size of the arrays of components queried must match.

Groups are used for many purposes, but the most important and least understood one is the customization of the memory layout. Entity components are in fact split according to the groups, therefore groups play an important role in the organization of the entity components and consequential way they must be iterated over.

if the two entities are built using a group for each entity descriptor, the entity components will be now laid out like:

which means that it’s now possible to iterate over the two entity views of E2, using the same index in the only loop necessary.

However as long as either GroupCompounds are used or the rule of one group for each EntityDescriptor is applied, the new Svelto user doesn’t really need to be aware of this architectural organization as it will just work.


New Feature: Find Groups

While more features can be added to make the advanced use of groups safer, FindGroups is already the solution for most of the cases. FindGroups returns all the valid groups were to find entities that generate the same set of components.

It’s simply used like this:

While FindGroups looks simple to use, the user shouldn’t rely on it as way to handle groups complexity.


New Feature: Engines Groups

Engines Groups may remember the ComponentSystemGroup found in UECS. Engines groups allow to group engines and even run them in a given order, using the SortedEngineGroup and the Sequenced engine attribute features. An example of how to sort engines using SortedEngineGroup is found in the MiniExample 1: Doofuses and Miniexample 2: Survival. The solution has been designed to allow engines from different assemblies (asmdefs in Unity) to be sorted without creating dependencies between assemblies. Only the Composition Root assembly will need to know the assemblies where the engines are declared.


New Feature: Native Memory and DOTS

Performance is obviously a hot topic for SECS. Even if Svelto.ECS has been designed to improve code maintainability, performance is one of the top priority for the framework. For Unity users, performance nowadays means using Jobs and Burst, which have been both designed to use native memory. While I have some personal strong opinions on how the Unity Jobs architecture has been designed, I consider Burst a marvel of technology. Using Burst fundamentally removes any needs to use native languages to seek high performance. Making SECS native memory compatible, especially since Unity mono doesn’t support Span<T> properly, has been an interesting challenge and at times frustrating. However, I am more than glad about the result achieved.

In these articles:

and

you will find the explanation of what I have done to support native memory in Svelto, which is necessary for Unity Jobs and Burst to work. Native memory will also give a little boost even when Jobs and Burst are not used. My special solution to support native memory has been tested successfully on several platforms. In general, the user doesn’t need to know when native memory or managed memory is used, but in SECS 3.0 EntityComponents are always stored in native memory, while EntityViewComponents are always stored in managed memory.

A set of native data structures have been added as well to work with native components. These are NativeBag, NativeDynamicArray and the very powerful SveltoDictionary. Since, at the moment, EntityComponents are always stored in native memory, these native data structures are necessary when it’s needed to store data in custom ways inside native components. However, since this is a necessity strictly linked to Unity DOTS, in future I will add the option to switch the storing of entity components to managed memory as well. All the native data structures explicitly created by the user must also be disposed of to avoid memory leaking.

The following Svelto.ECS operations are fully jobifiable and burstifiable:

  • Any structural change (Creation, Destruction and Swapping of entities)
  • Any entity components kind of query and their iteration
  • Being able to query entities by EGID (through NativeEGIDMapper and NativeEGIDMultiMapper)
  • Filters related operations

However note that while these operation can work in multi-threading code, svelto is not thread safe. It’s the user that must ensure that the operations executed on SECS DB are thread safe.

To see the power of fully jobified and burstified SECS, please have a look at MiniExample 1: Doofuses Iteration 3.

is this enough performance for you? (doofuses example)

New Pattern: integrate Svelto.ECS with Unity ECS

Svelto.ECS is designed to be platform agnostic. In fact, a bunch of Svelto users have already tested it on platforms outside Unity and thanks to the growing interested in using .net for game development, SECS can actually become an attractive solution to adopt the ECS paradigm everywhere. What about Unity though? Now that Unity ECS is becoming more stable, Unity users will absolutely wonder what’s the point in using Svelto.ECS. This is my take: UECS core has been initially designed to rewrite the Unity Engine features from Object Oriented code to Data Oriented Code. This process will take years to complete and UECS will evolve according to the new challenges the company will encounter. My conclusion is that UECS is fundamentally designed to develop an engine. At least this is my impression after using it in Gamecraft. Svelto.ECS has been designed instead just to develop games. Furthermore, I don’t plan to develop a 4.0 version. SECS will be feature complete during the 3.x cycle guaranteeing long term stability.

SECS shows its true power when it’s used together with a data-oriented engine that doesn’t use objects. That’s why SECS and UECS must actually operate together. Several Svelto.ECS features have been introduced to integrate UECS so to use pure ECS at every level. This integration has the benefit to keep UECS code simple and about what it has been designed for initially, that is handling the engine entities more than the game entities. Code maintainability and impressive performance can be easily achieved using SECS and UECS together. The logic is then simple: the user can see UECS as a data-oriented engine library instead than a game library and use Svelto on top of it, like if UECS is just the engine API.

MiniExamples 1: Doofuses (iteration 3), shows how SECS and UECS integration can be achieved, although it’s just one of the possible strategies.

While keeping SECS and UECS layers separate has some obvious code design benefits, it also introduces some drawbacks, although they didn’t weigh on us during the development of Gamecraft. These are mainly two: the need to synchronise entities between SECS and UECS and losing the automatic job dependency system resolution, which works only with UECS entity components. At Freejam, we haven’t found either of these drawbacks to be a noticeable problem.

At first, exposing UECS complexity may appear to be a drawback of the SECS and UECS integration. However this is in my opinion a benefit, as the automatic mechanisms in UECS while appealing at first, are awkward in complex projects scenarios.

However, the truth is that big companies that pay for unity support will be unlikely to consider using Svelto, as they can get direct support from Unity for every problem found. Albeit, Unity ECS is just a c# framework and as such, it would be a mistake to consider it the only option available as long as it is clear that integration is possible with its benefits and limits. The main reason I choose Svelto over UECS is its design approach which promotes the use of patterns that aid the writing of maintainable code.


Important Pattern: How to write abstract engines and reuse code

I am an advocate of introducing ECS as a paradigm and not a pattern. While one of the most obvious benefits of a 100% ECS application is that solutions look very similar to each other, ECS solutions to specific problems can be written with different strategies. However, this shouldn’t penalise code maintainability as it’s essential to reuse code written in engines so that they can operate on a not predefined super-set of entities. In other words, it is possible to write abstract ECS code. This is not the first time I expose this idea. In theory, using Inversion of Control, the code should be designed in multiple layers from the most abstract to the most specialised, following the SOLID Dependency Inversion Principle. Applying the principle with Svelto is simple: identify the behaviours to apply, identify the (abstract) entity this behaviour must be applied on. If the layer is abstract, the set of components may or may not identify a concrete entity. The layer can then be packaged, for example in a separate assembly, and expose Abstract engines, components, groups and entities descriptors that can be extended if necessary. Abstract layers serve the purpose to reuse as much code as possible without the user even realising it. Once the abstract engines are used, they will start to operate on the layer provided components.

In practice, the user must be aware that early abstraction is the root of all the evil. So unless the design is clear and firmly etched in the user mind, the best thing to do is to start working with specialised entities and engines and abstract only when shared/common behaviours between different entity types are identified.

There are several strategies to approach this problem:

The abstract layers provide components and group tags, possibly ExtendableEntityDescriptors too. The specialised layers are then expected to use GroupCompounds adopting the provided tags or extend entity descriptors from the provided ones.
In this scenario, the abstract engines iterate concrete entities found in agglomerated groups, like the DamageableEntitiesGroups example previously seen at the begin of this article.

Another strategy is for the abstract layer to not expect where the entity components will be found and use FindGroup to identify the list of groups where to find the superset of entities sharing a tuple of components. In this case the engine doesn’t even expect specific entities, but it iterates all the entities generated from entity descriptor sharing the same components.

Other strategies involving Filters and Generic Parameters implementing shared interfaces can be used by the resourceful user if they want to explore different patterns.

This paragraph deserves an entire separate article and hopefully it will come if requested by users.


Note: Svelto.ECS is not only Unity!

I understand why using SECS with Unity is controversial and I accept all the arguments. However let’s not forget that SECS works perfectly outside Unity too! It opens so many exciting opportunities since there are so many platforms supporting .net nowadays. This is a list of the ones I know about:

  1. Monogame
  2. Ports of XNA (FNA, FlatRedBall)
  3. Godot
  4. Stride
  5. Wave
  6. Unreal with UnrealCLR
  7. CryEngine
  8. UniEngine

The MiniExample 4 even shows how to use SECS with pure .NET and the SDL library and it’s the best starting point to experiment with SECS on platforms other than Unity. I am looking forward to integrating more not Unity examples in the MiniExample repository, but I need the community help for this.

Image

Side Note: Distribution and compatibility

Svelto.ECS for unity is distributed using OpenUPM or it can be directly downloaded from the Github repository. SECS uses the Microsoft Unsafe DLL which was supported by Unity until the previous version of DOTS. Since Unity dropped the support, I decided to distribute the Unsafe DLL through open UPM as well.

License wise Svelto.ECS is completely open to commercial use, modification and so on. There is no limit, I don’t need credits. Svelto community is small, but ever-growing so seeing more people joining us would surely be helpful for everyone.

For platforms outside Unity, my plan is to support the distribution through nuget ASAP.


Footnotes
0
Please leave a feedback on thisx

Please feel free to ask me what you want to read from my next articles. I need your feedback to know where I should spend my little spare time. Leave a comment or write on our discord channel! The future of Svelto.ECS development from now on will depend a lot by the users’ interest and contribution as well

5 2 votes
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments