(From WikiPedia) NorgesGruppen ASA is a Norwegian grocery wholesaling group which also runs various retail outlets. With a 38,5% market share in 2012, NorgesGruppen was the largest player in the Norwegian grocery retail market.

Norgesgruppen owns several store chains, like Kiwi, Meny, SPAR and others.

Executive summary

The several year long «Norgesgruppen project» encapsulated many things, native apps, web-sites, e-commerce and an entire server / client platform build from the ground up to serve data to all of the apps, websites and other solutions. My role as a lead in this project was to create, extend and maintain a service platform as well as advising and consulting with the chain teams.

My part in this

• Creating and maintaining the platform REST services
• Creating and maintaining the client SDK for apps and web-apps
• Creating and maintaining the platform realtime sync service
• Co-creating and maintaining the platform caching service
• Tech-lead, responsible for the platform servers and client SDK
• Advising and consulting chain teams

Tools and technologies used

• Node.js
• MongoDB / Mongoose
• Web sockets server and client side
• Azure WebApps
• Azure TableStorage, ServiceBus...and more
• Redis
• React.js
• TypeScript
• GitHub
• Elasticsearch
• Jira / Confluence
• TeamCity

Background

My involvement started pretty much at the beginning of the project. My role was to lay the foundation for several mobile apps by building a client SDK which would abstract away the underlying complexities of the humongous Norgesgruppen system. Since Norgesgruppen owns several store chains, each of the major chains had it's own team of designers and developers tasked with using this client SDK as a foundation and build chain specific experiences on top of it.

After some time the need for an entire platform criss-crossing all chains emerged. The overall goal of this platform was to make available both server and client APIs which would offer an accessible, consistent, scalable, secure and high-performance way for chain teams to access Norgesgruppen's data. We also wanted to add value to existing data via different logging, searching and data gathering methods which could be used by Norgesgruppen and their chains to better serve their customers. Lastly we where able to offer new functionality which did not exist within the existing Norgesgruppen services, like realtime shopping list synchronization and awesome search enhancements utilizing Elasticsearch.

Note that I'm not (at the time of writing this) personally an expert in the field of Elasticsearch, on this I depend on the excellent skills of my colleagues while I'm absorbing more knowledge about how to work with it.

REST / Sync services

Both the REST API, the caching services and the realtime sync services are build using Node.js and it's running on Azure WebApps.

The REST API is in place to act as a traffic buffer for Norgesgruppens own services and it's build to be easily scalable and extendable. The api provides custom services such as custom user data sync (not to be confused with the realtime sync services) as well as acting as cache layer enabling faster serving of data to clients.

The cache services are build on top of Redis and Elasticsearch. This system grabs data from underlying sources and serves it to client through the REST api, either as is or customized / optimized.

The realtime sync services is a high performance WebSocket based service used for instantly syncing data between many clients at once. It is a distributed system handling client sockets between many server instances and it uses a high performance queuing system to distribute messages between clients. This system also has a client side implementation provided as part of the client SDK (see below).

The client SDK

This is a JavaScript based SDK (written in TypeScript) used both by native iOS / Android applications as well as web based applications / web sites. For applications using this SDK it will handle all transactions with all of the servers mentioned above. In addition it will provide APIs that remain consistent over time, even when the underlying server api's are updated. It will also abstract away most complexities with storing data locally (IndexDB, WebSQL...) and many other nifty things.

Summary

Working with all of the talented people in this project has been and continues to be joy both from both a technical standpoint and a social one. Challenging tasks are commonplace and in this project this is a good thing. Working this way incrementally makes all of us more skilled and aware of what we are trying to accomplish, which is constantly improving on the user experience.