(From WikiPedia) NorgesGruppen ASA is a Norwegian grocery wholesaling group which also runs various retail outlets. With a 38,5% market share in 2012, NorgesGruppen was the largest player in the Norwegian grocery retail market.
Norgesgruppen owns several store chains, like Kiwi, Meny, SPAR and others.
The several year long «Norgesgruppen project» encapsulated many things, native apps, web-sites, e-commerce and an entire server / client platform build from the ground up to serve data to all of the apps, websites and other solutions. My role as a lead in this project was to create, extend and maintain a service platform as well as advising and consulting with the chain teams.
My part in this
Creating and maintaining the platform REST services
Creating and maintaining the client SDK for apps and web-apps
Creating and maintaining the platform realtime sync service
Co-creating and maintaining the platform caching service
Tech-lead, responsible for the platform servers and client SDK
Advising and consulting chain teams
Tools and technologies used
MongoDB / Mongoose
Web sockets server and client side
Azure TableStorage, ServiceBus...and more
GitHub - As an SCM and a project tool
Jira / Confluence
My involvement started pretty much at the beginning of the project. My role was to lay the foundation for several mobile apps by building a client SDK which would abstract away the underlying complexities of the humongous Norgesgruppen system. Since Norgesgruppen owns several store chains, each of the major chains had it's own team of designers and developers tasked with using this client SDK as a foundation and build chain specific experiences on top of it.
After some time the need for an entire platform criss-crossing all chains emerged. The overall goal of this platform was to make available both server and client APIs which would offer an accessible, consistent, scalable, secure and high-performance way for chain teams to access Norgesgruppen's data. We also wanted to add value to existing data via different logging, searching and data gathering methods which could be used by Norgesgruppen and their chains to better serve their customers. Lastly we where able to offer new functionality which did not exist within the existing Norgesgruppen services, like realtime shopping list synchronization and awesome search enhancements utilizing Elasticsearch.
Note that I'm not personally an expert in the field of Elasticsearch, on this I depend on the excellent skills of my colleagues while I'm absorbing more knowledge about how to work with it.
REST / Sync services
Both the REST API, the caching services and the realtime sync services are build using Node.js and it's running on Azure WebApps.
The REST API is in place to act as a traffic buffer for Norgesgruppens own services and it's build to be easily scalable and extendable. The api provides custom services such as custom user data sync (not to be confused with the realtime sync services) as well as acting as cache layer enabling faster serving of data to clients.
The cache services are build on top of Redis and Elasticsearch. This system grabs data from underlying sources and serves it to client through the REST api, either as is or customized / optimized.
The realtime sync services is a high performance WebSocket based service used for instantly syncing data between many clients at once. It is a distributed system handling client sockets between many server instances and it uses a high performance queuing system to distribute messages between clients. This system also has a client side implementation provided as part of the client SDK (see below).
The client SDK
Working with all of the talented people in this project has been and continues to be joy both from both a technical standpoint and a social one. Challenging tasks are commonplace and in this project this is a good thing. Working this way incrementally makes all of us more skilled and aware of what we are trying to accomplish, which is constantly improving on the user experience.