Software Caching involves storing frequently accessed data and code in a faster storage location to reduce latency and improve user performance and developer Experience. Here are the typical layers of software caching, based on common experiences and best practices.
In Memory Caching
NoSQL database often take advantage of this approach for archiving fast performance. I’ve seen and used this approach in several jobs and applicactions. Basically – Nomenclatures, data records that are not millions in count, that do not change – or do so – so rarely – it doesn’t matter – could be uploaded to RAM for optimal access. This also minimizes the need for querying them again from the database.
Application-Level Caching
This is application specific – in memory caching. Within an application, certain data or computations may be cached to improve performance. This could include caching frequently accessed database queries, computed results, or even entire objects to avoid expensive recalculations or database accesses.
Database Caching
Many database management systems (DBMS) support caching mechanisms to reduce the latency of database queries. This caching layer can cache frequently accessed data blocks, query results, or even entire tables in memory, reducing disk I/O and speeding up data retrieval.
Object Caching
Similar to Database – Object caching stores the results of expensive operations or frequently accessed data in memory. This can be implemented using dedicated caching systems like Memcached or Redis, which provide key-value stores optimized for fast data retrieval.
ORM
This level of caching is also used by Object Relational Mapping Libraries Like JPA and Hibernate. There is the notion of attached or detached entity (cached or not by the ORM). If there are multiple applications accessing the same database – this caching level should be taken special care. There are more than one cases in my experience when – records were not stored right away and the other app was not having access to data in real time.
Client-Side Caching
This caching occurs on the client side, such as in web browsers or mobile apps. There is the notion of Offline First Apps. It stores resources like web pages, images, and scripts locally on the user’s device. This reduces the need for repeated requests to the server, speeding up the overall browsing experience. You do this by registering JavaScript Service Worker.
Content Delivery Network (CDN)
A CDN caches static content, such as images, videos, and other media files, across distributed servers located closer to end-users. This reduces latency by serving content from servers that are physically closer to the user, improving overall performance and reliability. We think for the Internet as a Global Network, but sometimes – even the speed of light is introducing latency.
Proxy Caching
Proxy servers can cache content retrieved from web servers on behalf of clients. When a client requests the same content again, the proxy can serve it directly from its cache without needing to fetch it from the origin server, reducing bandwidth usage and improving response times.
- This is how DNS works. When you buy some domain – it is now live – for some user on the other side of the world right away
- Local Data Centers may implement this in an Application level for certain service. There is for example – requirement from the EU – all data to be stored inside an EU country.
Operating System Caching
Operating systems employ various caching mechanisms to improve performance at the system level. This includes caching frequently accessed files, directory structures, and even executable code in memory to reduce disk I/O and improve application responsiveness. When there is not enough RAM, the operating system uses some part of the ROM as such. This is often not a developer problem, but, you should think about it – if you run stuff self-suficiently yourself.
Container Caching
Docker and containerization is the current standard for backend applications. This may seen as another level of caching. Images are read-only templates – what the Application needs from the host operating system. When started – the System packages a container from the given instructions.
Container Binded Folders
Containers may be started with some folders from the Host System – binded as folders within the container. This way – even after delete or recreate – a container may start with additional data beyond the initial instructions like:
- database files
- CDN files
- bonus executable commands, apps or scripts
- folder where data will be kept even after shutdown
Deployment Caching
Over the years I’ve seen in the old world of monoliths somehow similar caching of libraries and dependencies. For example – Depending on the needs a typical Spring Boot App may get to 50-100mb. This may not seem as much for local development, but if this needs to be transfered over the Internet – it may slow things up. Instead, dependencies may be deployed once on the server (or via the binded folders) and the actual app will include only the essential (business logic) source code – and not all the million external code lines.
Downsides
The above may not work as expected with the newest ways of – packaging apps as native images. These are compiled binaries that do not need the Java runtime and get a super-fast start-up times.
Each of these caching layers plays a vital role in optimizing system performance and improving the developer experience, speed up feature delivery, minimize server downtime, improve user experience by reducing latency and minimizing resource utilization. However, it’s essential to carefully manage caching strategies to ensure data consistency, minimize cache invalidation issues, and prevent cache-related performance bottlenecks. One of the great questions of programming is – when/how to refresh the Cache.