Neo4j data processing with springboot with multiple database instance

Referring to my previous story on how to process Neo4j in springboot, I today underwent a unusual scenario on how can we build a ETL like functionality i.e. have 2 (or more) neo4j instances and use one for accessing metadata (like a central database) and use other for holding actual data.

When I searched Neo4j documentation, I could not find much examples on how can we configure multiple Neo4j instances using springboot. Springboot provides a built-in way to configure multiple database connections. In this post, I’ll do this with minimal configuration.

We’ll assume there are two Neo4j instances:

1. Neo4j1 — This holds central repository for customer data

1. Neo4j2 — This holds all the order related data

The problem is to get the customer_id from the request, get customer details (like name) from the central repo and then insert the order into Neo4j2

First thing I did is disabled auto-configuration for Neo4j:

package com.company.application;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.data.neo4j.Neo4jDataAutoConfiguration;
import org.springframework.data.neo4j.repository.config.EnableNeo4jRepositories;
@SpringBootApplication(scanBasePackages = {"com.company.**.**"})
@EnableAutoConfiguration(exclude = Neo4jDataAutoConfiguration.class)
@EnableNeo4jRepositories(basePackages = "com.company.neo4j.repo")
public class Application {
public static void main(String[] args) {
  SpringApplication.run(Application.class, args);
  System.out.println("Application is running...");
 }
}

Next, I added a configuration which creates beans for different Neo4j instances:

package com.company.application;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import lombok.Getter;
@Configuration
@Getter
public class Neo4jConfiguration {
@Value("${spring.neo4j1.uri}")
    private String neo4j1_uri;
@Value("${spring.neo4j1.authentication.username}")
    private String neo4j1_user;
@Value("${spring.neo4j1.authentication.password}")
    private String neo4j1_password;
    
    @Value("${spring.neo4j1.database}")
    private String neo4j1_database;
@Value("${spring.neo4j2.uri}")
    private String neo4j2_uri;
@Value("${spring.neo4j2.authentication.username}")
    private String neo4j2_user;
@Value("${spring.neo4j2.authentication.password}")
    private String neo4j2_password;
    
    @Value("${spring.neo4j2.database}")
    private String neo4j2_database;
@Bean
    @Qualifier("neo4j1")
    public Driver neo4j1Driver() {
        return GraphDatabase.driver(neo4j1_uri, AuthTokens.basic(neo4j1_user, neo4j1_password));
    }
@Bean
    @Qualifier("neo4j2")
    public Driver neo4j2Driver() {
        return GraphDatabase.driver(neo4j2_uri, AuthTokens.basic(neo4j2_user, neo4j2_password));
    }
}

Once done, we just autowired the same in the service class. This is how I get the customer name from the customer-id that was passed in the request:

private String getCustomerName(OrderCartInput req) throws Exception {
  String getCustomerMasterQuery = mapStringToMapping(cypherQueries.getCypher_queries().get("getCustomerMaster"), req);
  
  try (Session session = neo4jConfiguration.neo4j1Driver().session(SessionConfig.forDatabase(neo4jConfiguration.getNeo4j1_database()));
    Transaction tx = session.beginTransaction()) {
    Result r = tx.run(getCustomerMasterQuery);
    
    List<String> result = r.list().stream().map(temp -> temp.asMap().get("c.name").toString()).collect(Collectors.toList());
    
    if(result.size() != 1) {
     log.error("No customer found or more than 1 customer found for same customer id");
    } 
    else {
     log.info("Customer name --> " + result.get(0));
     return result.get(0);
    }
    
   } catch (Exception e) {
    log.error("Exception while processing data to Neo4j... ", e);
   }
  return null;
  
 }

And this is how I insert the data back in the 2nd Neo4j instance:

public String processOrder(OrderCartInput req) throws Exception {
  log.info("req --> " + req);
String createOrderQuery = mapStringToMapping(cypherQueries.getCypher_queries().get("createOrder"), req);
  createOrderQuery = createOrderQuery.replace("${name}", getCustomerName(req));
for (OrderItem item : req.getItems()) {
   String createOrderQuery1 = mapStringToMapping(createOrderQuery, item);
try (Session session = neo4jConfiguration.neo4j2Driver().session(SessionConfig.forDatabase(neo4jConfiguration.getNeo4j2_database())); 
    Transaction tx = session.beginTransaction()) {
    tx.run(createOrderQuery1);
    tx.commit();
   } catch (Exception e) {
    log.error("Exception while processing data to Neo4j... ", e);
   }
  }
return "Customer Order Processed Successfully";
}

My new cypher statements look like this now:

cypher_queries:
  getCustomerMaster: >
    MATCH (c:Customer{customerId: "${customerId}"}) RETURN c.name
  createOrder: >
    MERGE (c:Customer{name: "${name}", customerId: "${customerId}"})
    MERGE (o:Order{orderId: "${orderId}", orderDate: "${orderDate}"})<-[:PLACED]-(c)
    MERGE (oitem:OrderItem{item: "${item}", unit_price: "${unit_price}", quantity: "${quantity}"})<-[:HAS_ITEMS]-(o)

And finally, here is how I provide my configuration:

spring:
  neo4j1:
    uri: neo4j://localhost:7687
    database: test12
    authentication:
      username: neo4j
      password: test12
  neo4j2:
    uri: neo4j://localhost:7687
    database: test13
    authentication:
      username: neo4j
      password: test12
logging:
  level:
    root: INFO

I do not refute that there aren’t better ways, but this is one simple way to achieve this. Whole code can be found here.

Please do share your comments so that I can improve this.