I have read some pretty poor articles bashing Firestore recently. Generally they completely miss the features set or cargo cult Postgres SQL. This article attempt to highlight the features of Firestore that you won’t see with a Postgres solution (note I love Postgres), highlighting several area’s where Firestore is the world #1.

Clientside first

It’s designed for a direct connection to a mobile/webapp. This means it has a number of features that are unmatched in the market.

Latency compensation

Firestore maintains a local cache, so local writes and observable immediately. Greatly simplifying controller design. It even broadcasts writes to adjacent tabs in the browser…

Cloud Run is interesting, it’s a general-purpose elastic container hosting service, a bit like Fargate or Azure-Containers but with a few critical differences.

Most interesting is that it scales to zero, and auto-scales horizontally, making it very cost-effective for low traffic jobs (e.g. overnight batch processes).

It also runs arbitrary docker containers and can serve requests concurrently, meaning that for modest traffic you don’t usually need more than 1 instance running (and you save money).

Its flexibility comes at the cost of higher cold starts though. …

Recently Visnu Pitiyanuvath of Observable presented how dataviz techniques can be applied to developer dashboards to improve insights. I followed his advice and it had a transformational effect on my work.

(full talk is below but you don’t need to watch it right now)

His talk emphasized that we often over aggregate metrics [ proof]. Indeed, most monitoring dashboards are time series of the mean, perhaps some percentiles, but a load of line graphs nonetheless.

Typical Graphana dashboards are not good for building insight

His message was to stop doing that, and just draw every single event, no aggregation…

Over the last few months I have been solving a series of technical challenges that allows making BigQuery derived data visualisations public, completely within the browser, using free tools. The important thing is that your BigQuery credentials remain protected, yet you still do not need to run a server, or pay any money!

A powerful browser based literate coding and dataviz environment is provided by Observable.

My tutorial covers

  • Creating a Service Account
  • Password protecting it with with SubtleCrypto
  • Instantiating a BQ client in the browser using GAPI
  • Logging into Firebase Storage using a service account
  • Setting up Firebase Security Rules.
  • Reading and Writing to Firebase storage

Checkout the full tutorial here. It’s written in a notebook so the tutorial is executable and you can fork it to make your own BQ integration.

Originally published at https://dev.to on March 26, 2021.

I used to work at Google Cloud, I also worked on a Functions-as-service (FaaS) product (Firebase Functions). I have seen that customers kept complaining about the same things:

  • Deploy times are too long
  • Cold starts are too frequent
  • FaaS is hard to debug/mock

I then went on to work as Cloud consultant for all three major clouds, I also heard customers complain about the Cloud in general, particuarly

  • The unlimited risk of a big bill from Cloud providers.
  • The opaqueness of operation

As a consumer of digital services. I have have been outraged by

  • ISPs selling my data
  • Apps selling…

In this article, I take an in-depth look at using Camunda BPMN (Business Process Modeling Notation) to build a non-trivial internal tool for my workplace. Camunda BPMN process definitions are simultaneously a graphical and textual representation of an executable workflow. I find the technology very interesting as it works both a visual communication tool and as a low code rapid development system. It excels at long running operations and has a focus on reliability and observability. The supporting tooling that allows querying running processes both graphically and/or via REST APIs.

The tool I built here for evaluation purposes touches on…

A while ago I started writing an identity aware proxy (IAP) to secure a binary with authentication. However, what started as a minimal auth layer has grown with features. What I have come to appreciate is that the reverse proxy is a great layer to do a variety of cross-cutting concerns like auth, at-least-once delivery and adapting. Furthermore, I have found OpenResty provides amazing performance and flexibility, AND it fits the serverless paradigm almost perfectly.

Concretely I have been working on extending the IAP to ingest and reshape signals from Slack and Zapier, tunnel them through a Write Ahead Log…

A long time ago, our CEO used to have one-to-one discussions with every single Futuricean over the span of a calendar year. He did it to keep his finger on the pulse of what’s happening in the company and what people are thinking about. It was a great way to maintain an understanding of whether our autonomous organization was really sailing in the same direction and aligned behind a shared strategy. As organizations grow, we need to look for ways to scale the appropriate level of control and understanding. Our old tools are no longer effective.

That brings us to…

Seemingly every project I work on gets bogged down at some point on authentication. Why must it take so much time! Why must I solve the same problems so many times now! I found myself nodding furiously while reading Google’s BeyondCorp papers, “we should solve it in the proxy layer!”. Let’s stop wasting time with Oauth2 language clients, much like we did with TLS a while ago. Solve it well once, and reuse it for everything henceforth.

Why? Because Identity Aware Proxies are awesome for security, they allow you to take insecure binaries and wrap them with a pre-audited secure…

The goal I am working towards is a common language for designers and developers. The latest footstep is OpenAPI integration with BPMN diagrams, making it simpler to build human/service workflows by importing APIs from public specifications. Here is a snapshot of the latest demo prototype.

What is BPMN?

What you see is the Camunda Modeling programs view of a BPMN diagram. It’s a visual front end for an industry standard called Business Process Modeling Notation (BPMN 2.0). BPMN is powerful, it’s a flexible language for representing human/system interactions, with a number of implementations from different vendors (i.e. no vendor lock in). I am…

Tom Larkworthy

Cloud consultant at Futurice GmbH. Ex-Firebase, Ex-Google Cloud.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store