How to increase performance with fixed thread pools when retrieving bulk relational data

So, here my suggestion is to retrieve bulk data using multiple threads asynchronously and bind them together after complete the retrieval mechanism.

I will show you how can we achieve this using Java.

For this first you have to define number of thread we need to use. We can use a executor to adjust the thread pool size.

ExecutorService executor = Executors.newFixedThreadPool(2);

Now we can do the further implementation. I have two collections user and profile. I need to retrieve both user and profile details using two separate threads. In the above I passed 2 as thread pool size.

So these are the two collection retrievals I used for getting user and profile details.

code

private CompletableFuture<User> retrieveUser(ExecutorService executor) {

    return CompletableFuture.supplyAsync(() -> userRepository.findById(id).orElseThrow(() -> new RuntimeException()), executor);

}

private CompletableFuture<Profile> retrieveProfile(ExecutorService executor) {

    return CompletableFuture.supplyAsync(() -> profileRepository.findByUserId(userId).orElseThrow(() -> new RuntimeException()), executor);

}

supplyAsync means run each repository query in a separate thread from the thread pool.

CompletableFuture used to fetch data asynchronous.

Okay let's create the final functionality to retrieve full user details with user and profile collection details.

code

private UserDetailDTO getUserDetail() {

    try {



        // retrieve users

        CompletableFuture<User> userDetail = retrieveUser(executor);



        // retrieve profile

        CompletableFuture<Profile> profileDetail = retrieveProfile(executor);



        // combine results

        return CompletableFuture.allOf(userDetail, profileDetail)

                .thenApply(v -> {

                    User user = userDetail.join();

                    Profile profile = profileDetail.join();

                    return new UserDetailDTO(user, profile);

                }).join();



    } catch (Exception ex) {

        throw new RuntimeException(ex.getMessage());

    } finally {

        executor.shutdown();

    }

}

here actually the allOf() means waits for all futures to complete and finally we used join() to retrieve the results of each future.

As a last step we should need to shutdown the executor service. if not it might take a step to resource leak. So be sure to shutdown the executor after completes your work.

This is how final class methods should looks like.

code

public class UserDetails {

    public UserDetailDTO getUserDetail() {

        ExecutorService executor = Executors.newFixedThreadPool(2);

        try {

            // retrieve users

            CompletableFuture<User> userDetail = retrieveUser(executor);

            // retrieve profile

            CompletableFuture<Profile> profileDetail = retrieveProfile(executor);

            // combine results

            return CompletableFuture.allOf(userDetail, profileDetail)

                    .thenApply(v -> {

                        User user = userDetail.join();

                        Profile profile = profileDetail.join();

                        return new UserDetailDTO(user, profile);

                    }).join();

        } catch (Exception ex) {

            throw new RuntimeException(ex.getMessage());

        } finally {

            executor.shutdown();

        }

    }



    private CompletableFuture<User> retrieveUser(ExecutorService executor) {

        return CompletableFuture.supplyAsync(() -> userRepository.findById(id).orElseThrow(() -> new RuntimeException()), executor);

    }

    private CompletableFuture<Profile> retrieveProfile(ExecutorService executor) {

        return CompletableFuture.supplyAsync(() -> profileRepository.findByUserId(userId).orElseThrow(() -> new RuntimeException()), executor);

    }

}

It is a best practice use this methods inside a try catch block.

Thank you all have a nice day...........

Comments