Firebase strategy for @Shared property wrapper

[mortenbekditlevsen]

Dear all,

I've been wanting to use Firebase with TCA for a long time, but I haven't been able to see a good fit with real-time updating values and the Reducer until the great series about shared state.

Yesterday I started some experimentation with adding a firebase strategy, and I think that it could become a quite nice API.

My goals are:

• Using a similar or identical API to access data in Firestore and the Real-time database.
• Allow user to model a type-safe 'path' hierarchy that serves as a schema for your database(s)
• Model document and collection access differently in the path hierarchy, so a path to a document would allow synchronizing a single Codable entity and path to a collection would synchronize arrays, etc.
• When accessing collections, I would like to be able to make the API be read-only, since accessing collections are basically executing queries, and it may not make a lot of sense to 'write' back data for a query ((*) see comment below)
• Allow access to collections to be dictated by the type of the variable annotated with the property wrapper (for instance you may only be interested in an (ordered) array of model entities - or you may be interested in an array of (String, Entity) pairs in case you need to know the ids of the documents - or you may be interested in an unordered dictionary of keys and entities.)

(*) Could one imagine a read-only version of @Shared? I know it's not the basic use case for shared state, but for something like a live-updating query into some storage I think it could make sense. For now I just have an empty implementation of saving when dealing with collections.

Example of using the strategy:

First, you need to model the schema for your database:

extension FirebasePath where Element == FirebasePaths.Root {
    var chatrooms: CollectionPath<Chatroom> {
        append(["chatrooms"])
    }
}

extension FirebasePath where Element == Chatroom {
    var messages: CollectionPath<Message> {
        append(["messages"])
    }
}

This schema defines that in the root of your database, you have a collection of Chatroom entities at the path /chatrooms
A CollectionPath has a built-in child(_ id: String) method, that will give you an identified document in the collection.

In your Reducer's state, you could then have:

            @Shared(.firebase(FirebasePath().chatrooms.child("firechat"))) var firechat = Chatroom()

Which would synchronize the data in the path /chatrooms/firechat as a Chatroom entity into the firechat shared state variable.

In the following, the firechatMessages will be synchronized with all of the Messages located in /chatrooms/firechat/messages.

            @Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages)) var firechatMessages = []

Through the type-safe schema, the type of firechatMessages will automatically be inferred to be [Message].

It is possible to overload the strategy with multiple 'versions' of subscribing to a collection. For instance, if you want to also get the ids of the entities in your collection, you could use the overload:

            @Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages)) var firechatMessages: [KeyPair<Message>] = []

Where KeyPair is a generic that holds a String key and the generic value. This could be exposed as [(String, Message)] in the future, but currently that type is not Equatable (even though I expected (String, Message) to be implicitly Equatable due to SE-0283 ).
Well, I expect that could be fixed using variadic generics.

But the point is: Based on the shape of the collection you give to your shared data, you can control details about the synchronization.

Another overload could be made for unordered keys and values like this: (I haven't implemented this yet):

            @Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages)) var firechatMessages: [String: Message] = [:]

With regards to the goals:

Use same API for firestore and the realtime database

Not yet done.

The database type (together with other details like database id and region, in case you have multiple databases) could be encoded in the FirebasePath type, so that the schema defines where your data lives, and the user would not need to know or care.

Allow user to model a type-safe 'path' hierarchy that serves as a schema for your database(s)

Done

Model document and collection access differently in the path hierarchy

Done

Read-only access to collections

Well, currently it's just done with an empty implementation. This could be a fatalError or a runtime warning, but optimally it could be defined in the type system somehow.

Allow access to collections to be dictated by the type of the variable annotated with the property wrapper

Yup, that works!

Considerations

Type-safe paths are too verbose

When spelling out a type-safe path, it's currently done with FirebasePath().yourCollection.etc. The only public initializer for FirebasePath gives a FirebasePath<FirebasePaths.Root>, and this basically defines the root of your schema.

But I really don't like having to spell it out every time. If you could just do .yourCollection.etc it would look much nicer, but the FirebasePath can't be made static because the type actually carries state, and you could easily imagine having the same entity type stored in different locations, so in that case the paths can't statically be known from the entity type.

I hope that it would be possible to abbreviate this, to make initializing the firebase strategy look nicer.

Queries would be nice

Having access to a collection of data is fine, but the next step would probably be to add support for queries as well. Queries in Firestore and RTDB have different properties, so it would be interesting to see if those could be combined somehow.

Subscriptions to data are made when State is initialized

Perhaps it is not a real issue, but in case you initialize state for a lot of features directly from the entry point of your app, and each state perhaps has a lot of shared data subscriptions, this might cause a lot of data to start being synchronized.

In case this were an actual issue, perhaps some kind of delayed initialization of synchronization of shared data could be implemented?

[mortenbekditlevsen]

My experiments can be tried out here:

https://github.com/mortenbekditlevsen/swift-composable-architecture-firebase.git

[mortenbekditlevsen]

You can now unwrap a collection of data in Firestore as an IdentifiedArray. There are two overloads - if your entity is Identifiable, it will be unwrapped as IdentifiedArray<Entity.ID, Entity>, and if it's not, it will use the Firestore key as the id and unwrap as an IdentifiedArray<String, Identified<String, Entity>>.
Generics are magic! :-)

[mortenbekditlevsen]

Now the same API can be used for Firestore and RTDB.
It would be awesome if the user could easily opt out of either, but I'm not certain how to implement it.
I tried using #canImport, but it appears that I actually need to depend my library target on the actual firebase libraries for the #canImport to become true. And then everything is already decided in my Package.swift file.

[stephencelis]

@mortenbekditlevsen Thanks for sharing! We don't have a ton of Firebase experience, so we can't comment on some of the details, but to a few points:

When accessing collections, I would like to be able to make the API be read-only, since accessing collections are basically executing queries, and it may not make a lot of sense to 'write' back data for a query ((*) see comment below)
…
(*) Could one imagine a read-only version of @Shared? I know it's not the basic use case for shared state, but for something like a live-updating query into some storage I think it could make sense. For now I just have an empty implementation of saving when dealing with collections.

Our current recommendation with read-only is via access control:

public var firechatMessages: [Message] { _firechatMessages }
@Shared(.firebase(…)) private var _firechatMessages = []

But read-only has come up often enough that some kind of library API is under consideration.

You may find it helpful to look at a proof of concept for GRDB we pushed earlier:

shared-state-beta...shared-grdb

This is by no means production-ready and only means to simplistically drive the Todos demo, but it does demonstrate how local, mutable @Shared can be used to stage updates sent back to the database more explicitly.

Through the type-safe schema, the type of firechatMessages will automatically be inferred to be [Message].

Interesting! I found that this didn't work for me when dealing with an identified array, though. Did you have any trouble when adding support?

[mortenbekditlevsen]

Thanks a lot - I'll take a look at the grdb-branch. Otherwise, I guess that the read-only property is not a big deal. :-)

With regards to inference, I don't think that I did anything in particular to make it work.

For the three declarations:

@Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages)) var firechatMessages: IdentifiedArray = []
@Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages)) var firechatMessages2 = []
@Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages)) var firechatMessages3 = .init()
@Shared(.firebase(FirebasePath().chatrooms.child("firechat").massages)) var firechatMassages: IdentifiedArray = []

the inferred types are:

IdentifiedArray<String, Message>
[Message]
IdentifiedArray<String, Message>
IdentifiedArray<String, Identified<String, Massage>>

The difference between Message and Massage being that Message is Identifiable. Non-Identifiable types will be wrapped in an Identified<String, Thing> wrapper, since a document in Firestore or RTDB always have a String key.

I'm not sure why the inference behaves as it does. Array has an init, so why doesn't firechatMessages3 infer to the same as firechatMessages2?

I'll try and get some sample code up that uses the library.

[mortenbekditlevsen]

I couldn't find anything related to read only state in the branch shared-grdb, but I did have a look at the branch: shared-state-beta-readonly.

Was it the SharedReader from that branch you were thinking of?

That looks really interesting. Especially if a strategy through the type system could make itself conditionally 'compatible' with the Shared property wrapper and other times just with the SharedReader. So that the end user basically wouldn't be able to 'hold it wrong'. :-)

[mortenbekditlevsen]

Hmm, with regards to writing back to 'a collection of documents' or 'a query of documents', I guess that it could make sense. Especially with the ability to derive state for one element from the state of a collection.

It would mean that the saving of data should do something more intelligent than just writing each document back, but only if it had changes...

Changing documents in a collection makes sense. Deleting them also does.
Appending them is a bit more strange because is you have a query then the document you're appending might immediately change it's sorting order after creation due to the query...
Well... Hmm, perhaps it would work!

I guess it would mean that I should only support IdentifiedArray collections, since it would be hard to deal with diffs of plain arrays..

[mortenbekditlevsen]

Hi again @stephencelis ,

I just had time to experiment further with the released version of the shared state functionality.
What an elegant solution you found for handling read-only shared types!

I am now able to provide read-only overloads for the same strategy based on whether the type the user provides is Codable or only Decodable. It's so great!

It's still very much an experiment, but it is already very flexible in how it can be used.

[mortenbekditlevsen]

I implemented a small experiment allowing adding a query to a path. Just supports .limit(x) for now, and the implementation is not very nice, but it's just a proof-of-concept so far.

@Shared(.firebase(FirebasePath().chatrooms.child("firechat").messages.limit(1))) var firechatMessages: IdentifiedArray = []

• will only read the first document in the database.

[mortenbekditlevsen]

I've pushed a version that supports saving collections when they are IdentifiedArray collections.

Every document in Firestore or 'piece of data' in RTDB has a string key. Sometimes it makes sense to include that string key as an id in your data entity, but not always. This makes it not straight forward to figure out under what key to store that data.

If we have an IdentifiedArray<String, Identified<String, T>>, this means, that our entity itself has no id, so the 'id' of the Identified wrapper is exactly the key of the document in the database.

This means that you currently need to provide an id when adding a new entity to a collection. And you may actually not care about it. That needs to be considered somehow.

If we have an IdentifiedArray<T.ID, T> where T: Identifiable I currently build a map from T.ID to String upon synchronization, so that there is a disconnect between the id of the entity and the key under which it is stored. That's probably good to allow, since you may not want documents stored under their id. But it does require that you get an up-to-date value from the database before you write an update, because otherwise you don't have the map, and without the map it will currently be assumed that any new entity will be added to the database using an autogenerated key.

I think that the user could get full control of the latter by adding a protocol to your entity that means: My Identifiable.ID is a String and I would like to use that id as the key in the database...