Looking for engin.io replacement -joining our effort initiative

Charby

I am not sure if this is the right place for this topic...anyhow here are some feedback regarding my today progress :

Amazon : First, I have started to read the AWS documentation, but I found it very complex to dive in and determine what are the minimal sets of AWS module to implement ( do you have any recommandation about where to start ?). So I decided to put aside this option for now. Hope I will find something simpler to set up...
Google : then I switched to Google cloud and found a C++ google client SDK(great!) which I hopefull was able to build. Still after one hour of reading various documentation, I am still lost in the hundred of google API to figure out which one would best fit Qt for cross platform mobile application...so I decided once again to leave this option for a while...
Firebase (still google), I have started to evaluate Firebase which seems promising. I have check the REST API and it seems to be quite OK for NoSQL datastorage but :
- Firebase use PATCH http method which is not (yet) supported by Qt XMLHttpRequest. It might be possible to use x-http-method-override uri to overcome this limitation but so far I didn't succeed...
- I haven't been able neither to use .sv timestamp
- Firebase support asynchronous notificaiton through what they call "rest streaming" but I don't understand yet how to set it up
- Engin.io had a nice feature for creating server side thumbnails variant of uploaded pictures - I haven't see something similar with Firebase.
- it seems FB supports social login, but this is not straight forward using the REST api (once again I get lost on the required steps to achieve this...). This seems pretty simple with the SDK (javascript, android or iOS), too bad it doesn't propose a C++ sdk.

That's it for today, I will continue tomorrow, but I will be glad if meanwhile someone could share experience on this topic.

Charby

A bit more feedbacks :

I go on the evaluation of Firebase which seems quite matching my needs so far...but the REST API seems to be limited for now. I am considering starting a Qt wrapper around Firebase SDK. My plan would be to use the native Firebase SDKs depending of the OS and fall back using the REST api when the OS is not Android or iOS. In addition, I would like the wrapper to integrate a QabstractItemModel base that could be used from QML. Maybe this component could support others Backend...What do you think ?
I found this project an github which is a thin wrapper on Firebase REST API.
this is related topic, about BaaSBox support
additional Firebase ressources : mailing list & http://stackoverflow.com/questions/tagged/firebase related tag

JKSH

Hi @Charby,

I don't use cloud storage myself so I don't have much to contribute here, but I wanted to say thanks for starting this initiative. I hope others who are interested will join in, so that everyone benefits.

gadlim

I'm also a Qt CS user (Enginio + MWS), and evaluating other backends.
Firebase has some nice features, but the query API is very basic (no OR, can't chain AND,...). Another problem is that if you want fast data access, you have to use their SDK, the REST API works but is a lot slower. It's also a complete lock-in to the service.
Baasbox looks like a small(ish) project, as do most others (deployd, remotestorage.io, ...).

The best contender I've found so far is IBM Cloudant:

full featured (lots of features)
scalable
can be self-hosted
language-agnostic: REST API is not second-class
mostly OSS (CouchDB-based), less lock-in that you could imagine from IBM

benlau

How about Amazon API gateway + Lambda? You could code in Javascript and don't need to maintain your own server.

Microservices without the Servers | AWS Compute Blog

Charby

@benlau that could be indeed another good candidate to evaluate...

gadlim

@benlau AWS Lambda is for creating your own "micro-services" (~API), that can be useful, but it's more comparable to the MAR part of Qt CS than to Qt EDS (a.k.a. Engin.io).

bnogal

i just post to say, i support the idea!

lets see what i get from my little code. i hope i can upload it soon this week.

gadlim

Follow-up on my Cloudant evaluation: I ran into problems trying to figure out how to encrypt user passwords.
I'll have a conversation on that subject with IBMers tomorrow.
I hope to also discuss the broader subject of whether Cloudant is a good fit for 2-tiers apps (no server code, or almost no server code).

If you guys have questions, tell me, I'll try to insert them in the conversation.

bnogal

Well

I am almost ready to upload my code.
I just miss a class... that i used lately at my job and i should remove it from my personal code now...

It is a swissknife class that it is almost no needed, but does a basic task.

For the login.

My code uses google login in android, so security is taken by google and baasbox. It should be possible to do something similar for iphone.

For "secure" password login without using "social login" by the moment i just hash at application level the password, as by the moment i dont use ssh for requests. It doesnt make secure the system, but unless it doesnt exposes the original password in plain text.
It should be easy to improve the code to use a certificate and SSL sockets with some Qt magic.

Today i have a some social event... hope i can rip off that class and upload it tomorrow. The rest is ready

Charby

@bnogal Sounds great !
From my side, I couldn't progress at that rate...what I did is create the skeleton of a plugin (I was unfamiliar with the plugin mechanism) and request for a playground project from the developer mailing list. I have continued evaluation with Apigee candidate. But I feel I should restrain myself now and focus on a something simple that could be later on improved so I will start with the firebase plugin. Hopefully I should be able to publish in a couple of days.

gadlim

Hello,

Quick update from Cloudant:

Security should be ok, but I'm still missing some pieces
Offline+sync support for mobile and web apps with PouchDB, very similar to how the Firebase SDK works (but with more options for queries, including server-side filters)

xargs1

Looking at people's attempts to get PouchDB working with Qt/QML, it doesn't seem very encouraging.

gadlim

@xargs1 said:

Looking at people's attempts to get PouchDB working with Qt/QML, it doesn't seem very encouraging.

Do you have some links ?
There's a Python adapter project that uses Qt internally to wrap PouchDB, so a C++/QML adapter should be feasible. That said, maybe it's too much effort and we should to stick to the REST API.

xargs1

As far as I'm aware, PouchDB is pure JavaScript and doesn't have a REST API.

Here's the discussion on failed attempts to integrate PouchDB into QML:

https://github.com/pouchdb/pouchdb/issues/3709

gadlim

@xargs1 said:

As far as I'm aware, PouchDB is pure JavaScript and doesn't have a REST API.

I was referring to the Cloudant REST API

Here's the discussion on failed attempts to integrate PouchDB into QML:

https://github.com/pouchdb/pouchdb/issues/3709

I see. It's in a "maybe could work with some efforts but nobody's working on it" state.

xargs1

It seems to be like most of the large JavaScript frameworks: it assumes it's running in either a browser or node.js, and QML's JavaScript doesn't provide either of those environments.

gadlim

@xargs1 said:

It seems to be like most of the large JavaScript frameworks: it assumes it's running in either a browser or node.js, and QML's JavaScript doesn't provide either of those environments.

Yes.
There's another non-JS (Android) adapter that's using the same tactics as the PyQt adapter (hidden Webview), but it's also unmaintained.
What's strange is that IBM (Cloudant) is advertising PouchDB as the right solution for mobile apps sync, so they're focusing on "mobile" web apps maybe.

Charby

Hi all,

I think it would be helpful to create a Qt playground project so we could publish our backend wrappers in a commun repository. I have sent a request for creating the playground project to the development mailing list but I don't know whether it is the way to go...Do you know what is the process to follow to create a playground project ?

So far, I have designed a QML plugin which is a work in progress :

the plugin only offers for now a Firebase backend element which is quite limited : the only supported authorization method is the secret token id and at this time it is only a thin wrapper around the REST API (GET, DELETE, POST, PATCH and PUT are supported).
what remains to be done :
- define and implement an API suitable for supporting multiple backend : one option could be to design an API similar to the former enginio (create, update, query, uploadFile, downloadURL, remove...) but this API relies on backend features that we would have to mimics (users and usergroups, fileref...) so I am not sure this would be the way to go...
- create a model that could be nicely used from QML like the former enginio model
- use the Android and iOS sdk depending on the deployed platform (and using the REST sdk otherwise)
- implement other backend using the same API (cloudant, apigee, BaaSBox...)

I don't know if this does worth to publish this to github or to wait for the creation of a playground qt module...

gadlim

@Charby said:

I don't know if this does worth to publish this to github or to wait for the creation of a playground qt module...

I'd say, if you have some code, even WIP, publish it. When discussing specs in a group, actual code can help to keep everyone on the same page.

Given our limited resources, it would help a lot to have only a thin layer of code to add to another, wider audience project (at the moment Qt/QML on mobile is a small audience, sadly). I thinking specifically of Hood.ie. I don't know if it's technically feasible, but using Hoodie as a kind of middleware could save a lot of code; just mimicking its API could be useful too. Of course if (for instance) Hoodie is bound to the browser, we have the same problem as with PouchDB.