MICROSERVICES - Node.js

Introduction to microservices

What is Microservice ?

An architectural style where an application is divided into small, independent services that communicate via APIs
Characterstics of microservices
- Independent deployment : you can update and redeploy only one service without affecting others.
- Loose coupling: Each service (auth, payments, orders) is independent and communicates via APIs
- Polyglot (different services can use different tech)
- Scalability per service
- We also use diffrent tech stack in diffrent services
Difference from Monoliths:
- Monolith = one big codebase, tightly coupled(All modules inside one code base)
- Microservices = many small services, loosely coupled(Each service is independent)

CORE CONCEPTS

Service = A small, independent unit (e.g., User Service, Order Service).

API Gateway = A single entry point to route requests to services.
Service Registry = Keeps track of services (like Consul, Eureka).
Communication:
- Sync: REST (HTTP/Express), gRPC
- Async: Message Queues (RabbitMQ, Kafka, Redis Pub/Sub)
Database per Service:
- Each microservice should manage its own database.

HOW THE SERVICE INTERACT WITH API-GATEWAY

Create api-gateway and identity-service two folders in your main root project folder

// api-gateway/src/server.js
npm i cors dotenv express express-http-proxy helmet ioredis jsonwebtoken winston nodemon

//idetity-service/src/server.js
npm i argon2 dotenv express express-rate-limit helmet ioredis joi jsonwebtoken mongoose rate-limit-redis winston
npm i rate-limiter-flexible
//argon2 -> for password hasing(better than bycrypt)
//winston -> logging library for Node.js

Identity service → Our this service handle authentication and authorization features , we simply treat this as standalone Node.js application

 //1. Create User Model with pre middleware that uses argon2 for password hasing 
 //2. Create logger.js in utils for winston
 //3. Create errorhandler.js in middlware folder
 //4. Create RefreshToken: Long-lived, used to request new access tokens without forcing the user to log in again
 // Think of it like:
 //🎫 Access Token = Movie ticket (valid for 3 hours, then it expires).
 //🪪 Refresh Token = Membership card (long-term, can get you new tickets anytime).
 //5. Crate userRagistration validation with joi and signup with accessToken and refreshtoken
 //6. Implement rate-limiter-flexible with redis in server.js
 //7. Implement express-rate-limiter for sensistive endpint in server.js
 //8. Finallly listen server test endpointt in postman

Connect identity service to api gateway

Instead of calling each service directly → use API Gateway.

 //1. Make logger and errorHanler like identity-service

 //2. Chnage in env file 
 // PORT=3000
 // NODE_ENV=development
 // IDENTITY_SERVICE_URL=http://localhost:3001
 // REDIS_URL=redis://localhost:6379

 //3.Craete rate express rate limiting same as identity in server.js

 //4. CREATE PROXY and //Proxies it to IDENTITY_SERVICE_URL (e.g., http://localhost:3001) , Incoming: /v1/auth/register , Outgoing: /api/auth/register

Example Flow of identity service:
1. User logs in → server generates access token (1h) + refresh token (7d).
2. User calls APIs with access token.
3. After 1h → access token expires → user gets 401 Unauthorized.
4. Client automatically sends refresh token → server verifies → issues a new access token.
5. If refresh token is invalid/expired → user must log in again.

Post-service

 // rounning on PORT = 3002
 // In server.js 
 ////routes: also pass our redis client -> this private route prootected by auth middleware
 app.use('/api/posts',  (req,res,next)=>{ //Adds the redisClient to every request object (req.redisClient).
     req.redisClient = redisClient  //So inside your route files (post-routes.js), you can directly access Redis:
     next()
 }, postRoutes)

Connect Post Service to api gateway

 //1. Make auth middleware -> that validate our token
 //2. In server.js setup proxy
 //setting up proxy for post service
 app.use('/v1/posts' ,auth ,  proxy(process.env.POST_SERVICE_URL , {
     ...proxyOptions,
     proxyReqOptDecorator:(proxyReqOpts , srcReq)=>{
         proxyReqOpts.headers["Content-Type"] = "application/json";
         proxyReqOpts.headers['x-user-id'] = srcReq.user.userId; //So the Post Service knows which user made the request, without decoding the JWT again.
         return proxyReqOpts
     },
     userResDecorator:(proxyRes, proxyResData , userReq , userRes)=>{ //modifies/logs the response before sending back to client.
         logger.info(`Response received from Post Service:${proxyRes.statusCode}`)
         return proxyResData;
     }

 }))

Do Caching while fetching all posts

     try{

         //do pagination 
         const page = parseInt(req.query.page) || 1;
         const limit = parseInt(req.query.limit) || 10;
         const startIndex= (page-1) * limit;

         //do caching
         const cacheKey = `posts:${page}:${limit}`
         const cachedPosts = await req.redisClient.get(cacheKey) // as we pass post in redisclient in server.js

         if(cachedPosts){
             return res.json(JSON.parse(cachedPosts))
         }

         // if cachedPost not present in redist  we fetched them from DB
         const posts = await Post.find({}).sort({createdAt:-1}).skip(startIndex).limit(limit) 
         const totalPosts = await Post.countDocuments();

         const result={
             posts,
             currentPage:page,
             totalPages: Math.ceil(totalPosts/limit),
             totalPosts:totalPosts
         }

         // save your post in redis cache
         await req.redisClient.setex(cacheKey , 300 , JSON.stringify(result)) // store for 5 mintutes

         return res.json(result)
     }

 //Now see the time in post its coming from cahchee

Must delete cache while creating new post , delete any post

 //delete redis keys: use this when we create new post or update any post or change data of any post then we must delete that post from cache
 async function invalidatePostCache(req, input){
     const cachedKey = `post:${input}` // for deletepost
     await req.redisClient.del(cachedKey)
     const keys = await req.redisClient.keys("posts:*")
     if(keys.length>0){
         await req.redisClient.del(keys)
     }
 }

Media Service:

 // setting up proxy for media service
 app.use('/v1/media' , auth , proxy(process.env.MEDIA_SERVICE_URL , {
     ...proxyOptions,
     proxyReqOptDecorator:(proxyReqOpts , srcReq)=>{
         proxyReqOpts.headers['x-user-id'] = srcReq.user.userId;
         /*
         ✅ Ensure the Content-Type is correctly set:
         - If the incoming request is *NOT* a file upload (`multipart/form-data`),
           then force the Content-Type to `application/json`.
         - This avoids corrupting file uploads by interfering with form data boundaries.
       */
         if(!srcReq.headers['content-type'].startsWith('multipart/form-data')){
             proxyReqOpts.headers["Content-Type"] = "application/json";
         }
         return proxyReqOpts
     },
     userResDecorator:(proxyRes, proxyResData , userReq , userRes)=>{ //modifies/logs the response before sending back to client.
         logger.info(`Response received from media Service:${proxyRes.statusCode}`)
         return proxyResData;
     },
     parseReqBody:false  //Without parseReqBody: false, the proxy would:Try to parse photo.jpg as text or JSON → ❌ broken.

 }))

To communicate between two services we use - RabbitMQ

RabbitMQ is a message broker — it acts like a middleman between different parts (microservices) of your application.

RabbitMQ- It allows asynchronous communication between services — meaning one service can send a message, and the other service can process it later, without both being connected at the same time.
Imagine:
- You (Media Service) write a message saying “Hey, new media uploaded with ID=123”.
- You drop it into a mailbox (RabbitMQ).
- The Post Service opens the mailbox, reads the message, and updates its database.

Problem: We want to delete the post from post service but the media Ids inside post came from media service so if we delete the post from post-service we also need to delete the media from media-service, To make a connection between both services we use - RabbitMQ

INSTALLATION:

Install - https://www.erlang.org/downloads 
Install- https://www.rabbitmq.com/docs/install-windows
Now get the Url: C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.4\sbin
and pase it on cmd like this : > cd C:\Program Files\RabbitMQ Server\rabbitmq_server-4.1.4\sbin
Now: >rabbitmq-plugins enable rabbitmq_management
npm i amqplib -> Install this in both post and media service , it as the language (protocol) that RabbitMQ and other messaging systems use to send and receive messages between services.

SETUP:

//post-service
//In .env
RABBITMQ_URL=amqp://localhost:5672
//In utils create - rabbitmq.js (make connection)
const amqp = require('amqplib')
const logger = require('./logger')
let connection = null;
let channel = null;
const EXCHANGE_NAME = 'facebook_events'
async function connectionRabbitMQ(){
    try{
        connection = await amqp.connect(process.env.RABBITMQ_URL);
        channel = await connection.createChannel();
        await channel.assertExchange(EXCHANGE_NAME , "topic" , {durable:false})
    }
    catch(err){
        logger.error('Error connecting to rabbit mq' , e);
    }
}

async function publishEvent(routingKey , message){
    if(!channel){
        await connectionRabbitMQ()
    }
    channel.publish(EXCHANGE_NAME, routingKey , Buffer.from(JSON.stringify(message)))
    logger.info(`Event published: ${routingKey}`)

}
module.exports = {connectionRabbitMQ , publishvent};

Connect RabbitMQ in server.js

async function startServer(){
    try{
        await connectionRabbitMQ();
        app.listen(PORT , ()=>{
          logger.info(`Post Service is Running at PORT:${PORT}`)
        })
    }
    catch(err){
        logger.error('Failed to connect Server',err)
        process.exit(1)
    }
}
//listen server
startServer();

delete post from post service

// delete controller
        //publish Event: Here we publish event via rabbitmq this event is consume by the media service to delete media
        await publishEvent("post.deleted", {
            postId: post._id.toString(),
            userId: req.user.userId,
            mediaIds: post.mediaIds,
        })

Consume from media service

//1. Create connection of rabbitmq like above post service 
//2. Write consume method like publish method
async function consumeEvent(routingKey , callback){
    if(!channel){
        await connectionRabbitMQ();
    }
    const q = await channel.assertQueue("",{exclusive:true});
    await channel.bindQueue(q.queue , EXCHANGE_NAME , routingKey);
    channel.consume(q.queue , (msg)=>{
        if(msg !=null){
            const content = JSON.parse(msg.content.toString());
            callback(content)
            channel.ack(msg)
        }
    })

    logger.info(`Subscribed the event: `, routingKey)
}
module.exports = {connectionRabbitMQ , publishEvent ,consumeEvent};

In server.js

//listen server
async function startServer(){
    try{
        await connectionRabbitMQ();
        //consume events
        await consumeEvent('post.deleted' , handlePostDeleted) // pass  routeringKey same as post service , call a event hanlder 
        app.listen(PORT , ()=>{
          logger.info(`Media Service is Running at PORT:${PORT}`)
        })
    }
    catch(err){
        logger.error('Failed to connect Server',err)
        process.exit(1)
    }
}
//listen server
startServer();

Create Event Handlers in media service

When we delete a post from post service it sent or publish the data to the media service then media service use or consume that data and delete that media from cloudnary

Data publish look like :

// We create this data above 
// This is my event
{
  postId: '68ea2bbf44b8995069294c77',
  userId: '68dc167eb994c9ac200bdfbf',
  mediaIds: [ '68ea2b956627018bc614b978' ]
}
// Now use this media id in media-service to delete media
const Media = require("../models/Media");
const logger = require("../utils/logger");
const { deleteMediaFromCloudinary } = require("../utils/mediaUploader");

exports.handlePostDeleted = async(event)=>{  // consume this above
    console.log(event, "eventeventvent");
    const{postId ,mediaIds} = event
    try{
        const mediaToDelete = await Media.find({_id:{$in:mediaIds}})
        for(const media of mediaToDelete){
            await deleteMediaFromCloudinary(media.publicId)  // delete from cludinary
            await Media.findByIdAndDelete(media._id) // delete from database
            logger.info(`Deleted media ${media._id} associated with deleted post ${postId}`)
        }
    }
    catch(err){
        logger.error("Error While deleting media in media service",err)
    }
}

Search Service:

When our user create a new post from post-service then it publish a event now this event is consumed by our search service and store the same post in the search-service.

We dont need to create other route to create post in search service just use publish event and consume using rabbitmq

MICROSERVICES - Node.js

Introduction to microservices

CORE CONCEPTS

HOW THE SERVICE INTERACT WITH API-GATEWAY

Example Flow of identity service:

Comments

More from this blog

9. Redis - Node.js

8. TypeScript - In Node.js

7. GraphQL - Node.js

6. Socket.io - Node.JS

Command Palette

Introduction to microservices

CORE CONCEPTS

HOW THE SERVICE INTERACT WITH API-GATEWAY

Example Flow of identity service:

Comments

More from this blog