Skip to main content

How to increase performance with fixed thread pools when retrieving bulk relational data




So, here my suggestion is to retrieve bulk data using multiple threads asynchronously and bind them together after complete the retrieval mechanism.

I will show you how can we achieve this using Java.

For this first you have to define number of thread we need to use. We can use a executor to adjust the thread pool size.

ExecutorService executor = Executors.newFixedThreadPool(2);

Now we can do the further implementation. I have two collections user and profile. I need to retrieve both user and profile details using two separate threads. In the above I passed 2 as thread pool size.

So these are the two collection retrievals I used for getting user and profile details.

private CompletableFuture<User> retrieveUser(ExecutorService executor) {

    return CompletableFuture.supplyAsync(() -> userRepository.findById(id).orElseThrow(() -> new RuntimeException()), executor);

}

private CompletableFuture<Profile> retrieveProfile(ExecutorService executor) {

    return CompletableFuture.supplyAsync(() -> profileRepository.findByUserId(userId).orElseThrow(() -> new RuntimeException()), executor);

}


supplyAsync means run each repository query in a separate thread from the thread pool.

CompletableFuture used to fetch data asynchronous.

Okay let's create the final functionality to retrieve full user details with user and profile collection details.

private UserDetailDTO getUserDetail() {

    try {


        // retrieve users

        CompletableFuture<User> userDetail = retrieveUser(executor);


        // retrieve profile

        CompletableFuture<Profile> profileDetail = retrieveProfile(executor);


        // combine results

        return CompletableFuture.allOf(userDetail, profileDetail)

                .thenApply(v -> {

                    User user = userDetail.join();

                    Profile profile = profileDetail.join();

                    return new UserDetailDTO(user, profile);

                }).join();


    } catch (Exception ex) {

        throw new RuntimeException(ex.getMessage());

    } finally {

        executor.shutdown();

    }

}


here actually the allOf() means waits for all futures to complete and finally we used join() to retrieve the results of each future.

As a last step we should need to shutdown the executor service. if not it might take a step to resource leak. So be sure to shutdown the executor after completes your work.

This is how final class methods should looks like.

public class UserDetails {

    public UserDetailDTO getUserDetail() {

        ExecutorService executor = Executors.newFixedThreadPool(2);

        try {

            // retrieve users

            CompletableFuture<User> userDetail = retrieveUser(executor);

            // retrieve profile

            CompletableFuture<Profile> profileDetail = retrieveProfile(executor);

            // combine results

            return CompletableFuture.allOf(userDetail, profileDetail)

                    .thenApply(v -> {

                        User user = userDetail.join();

                        Profile profile = profileDetail.join();

                        return new UserDetailDTO(user, profile);

                    }).join();

        } catch (Exception ex) {

            throw new RuntimeException(ex.getMessage());

        } finally {

            executor.shutdown();

        }

    }


    private CompletableFuture<User> retrieveUser(ExecutorService executor) {

        return CompletableFuture.supplyAsync(() -> userRepository.findById(id).orElseThrow(() -> new RuntimeException()), executor);

    }

    private CompletableFuture<Profile> retrieveProfile(ExecutorService executor) {

        return CompletableFuture.supplyAsync(() -> profileRepository.findByUserId(userId).orElseThrow(() -> new RuntimeException()), executor);

    }

}


It is a best practice use this methods inside a try catch block.

Thank you all have a nice day...........

Comments

Post a Comment

Popular posts from this blog

Building a Resilient Upsert in Spring Boot

I got this idea when I was working on a project that deals with a third party API. Actually my requirement was to update the database everytime if there is any new change with my third party API. So, it comes like this. I had a project that needed to sync the country list from a third party API. It is very simple as you know and the countries are not going to change in future. It is most likely the same country list. But the issue comes now. Sometimes they remove some countries from their list because they stop supporting that specific country. (Think the API is not providing a webhook). So, now what can I do? Can I use the same country list and throw errors from my system everytime. No, I had to think about a solution to update the country list every time. When I was thinking about a way to update the list, I found a better and efficient way to solve this problem. So, I will explain the steps I followed to solve this issue. Before that you can ask why you can’t keep the country list r...
Post a Comment
Read more

How to create cloned object using Java

I have an experience with updating a user profile with checking unique email and phone number. In that case I want to get my previous profile details and want to set new updated object details. But after I set the new object details to my retrieved previous object, my previous object setting with new data. So I have a way to again recall the previous data object using a database call.  But if you prefer to maintain a performance based app, you can reduce the database call using cloning the objects. Here actually we create the same object as old and new two objects.  First I will show how we can achieve using Java 8 version. Think if you have two classes Profile and DomainEntity. Here actually DomainEntity used for extend generic objects like created date, updated date etc. @Getter @Setter public class DomainEntity {     private Date createdDate;     private Date updatedDate;     private boolean deleted; } @Getter @Setter public class Profile exten...
Post a Comment
Read more

Better Way to Make a Shared Data Grid Using Material UI and React

Sometime ago I have a requirement to make a shared data grid that can use inside different components in the react. I tried with different scenarios but couldn’t figure out a better way to make a shared data grid with different options. My requirement is also gather with Typescript and Material UI version 5.0. So I used their latest Material data grid free release(version 6). When we plan to use v6 data grid, It is bit different than version 5. So I like to show how I fulfill my requirements and how I made my components in a better way. Ok let’s go. First we need to have latest react version and Material UI version up to 5. Then in here I’m planning to work on Material UI x-data-grid version 6. In here I’m using Typescript to show the coding examples. Ok, Let’s create our shared data grid components like below. I will explain about some properties I have used inside this data grid Material UI component. This is my basic setup of data grid component. import React from 'react'...
Post a Comment
Read more