DEV Community

loading...
Cover image for How to develop an online code compiler using Java and Docker

How to develop an online code compiler using Java and Docker

zakariamaaraki profile image Zakaria Maaraki Updated on ・5 min read

In this tutorial, we are going to see an overview of how to create a simple and efficient online code compiler for competitive programming and coding interviews using Java (Spring Boot) and Docker.

Before starting you can find the source code of the project at the following link https://github.com/zakariamaaraki/RemoteCodeCompiler

Why Java and Docker?

Well for java it's a personal choice, I really like working with this language, but you can choose any other programming language you prefer.

To separate the execution environments from different source codes (to avoid malicious code affecting your machine) and to limit resources, we need to use virtual machines or containers, but what is the difference between these two choices?

Virtual machine vs container

Alt text

A virtual machine (VM) is an emulation of a computer system. Put simply, it makes it possible to run what appear to be many separate computers on hardware that is actually one computer.

With containers, instead of virtualizing the underlying computer like a virtual machine (VM), just the OS is virtualized.

Containers sit on top of a physical server and its host OS — typically Linux or Windows. Each container shares the host OS kernel and, usually, the binaries and libraries, too. Shared components are read-only.

So now it's clear that using containers it's a better choice for us as they are lighter than Vms, and docker is the most used containerization solution, that's why i chose docker for this project.

The API

Alt text

First, we need to design the API, let's say we want to provide an online compiler for 4 programming languages (Java, C, C++ and Python).
So the API should look like something like this :
Four controllers one for Java, one for C, one for C ++ and another for Python.
The call to these controllers is done through POST requests to the following urls :

  • localhost:8080/compiler/java
  • localhost:8080/compiler/c
  • localhost:8080/compiler/cpp
  • localhost:8080/compiler/python

As inputs, we are expecting 5 fields :

  • output : the expected output.
  • sourceCode : the source code in java, c, c++ or python.
  • timeLimit : the time limit in seconds that the source code must not exceed during its execution (must be between 0 and 15s).
  • memoryLimit : the memory limit in Mb that the source code must not exceed during its execution (must be between 0 and 1000Mb).
  • inputFile : inputs written in separate lines (optional).
// Python Compiler
    @RequestMapping(
            value = "python",
            method = RequestMethod.POST
    )
    public ResponseEntity<Object> compile_python(@RequestPart(value = "output", required = true) MultipartFile output,
                                            @RequestPart(value = "sourceCode", required = true) MultipartFile sourceCode,
                                            @RequestParam(value = "inputFile", required = false) MultipartFile inputFile,
                                            @RequestParam(value = "timeLimit", required = true) int timeLimit,
                                            @RequestParam(value = "memoryLimit", required = true) int memoryLimit
    ) throws Exception {
        return compiler(output, sourceCode, inputFile, timeLimit, memoryLimit, Langage.Python);
    }

    // C Compiler
    @RequestMapping(
            value = "c",
            method = RequestMethod.POST
    )
    public ResponseEntity<Object> compile_c(@RequestPart(value = "output", required = true) MultipartFile output,
                                          @RequestPart(value = "sourceCode", required = true) MultipartFile sourceCode,
                                          @RequestParam(value = "inputFile", required = false) MultipartFile inputFile,
                                          @RequestParam(value = "timeLimit", required = true) int timeLimit,
                                          @RequestParam(value = "memoryLimit", required = true) int memoryLimit
    ) throws Exception {
        return compiler(output, sourceCode, inputFile, timeLimit, memoryLimit, Langage.C);
    }

    // C++ Compiler
    @RequestMapping(
            value = "cpp",
            method = RequestMethod.POST
    )
    public ResponseEntity<Object> compile_cpp(@RequestPart(value = "output", required = true) MultipartFile output,
                                            @RequestPart(value = "sourceCode", required = true) MultipartFile sourceCode,
                                            @RequestParam(value = "inputFile", required = false) MultipartFile inputFile,
                                            @RequestParam(value = "timeLimit", required = true) int timeLimit,
                                            @RequestParam(value = "memoryLimit", required = true) int memoryLimit
    ) throws Exception {
        return compiler(output, sourceCode, inputFile, timeLimit, memoryLimit, Langage.Cpp);
    }

    // Java Compiler
    @RequestMapping(
            value = "java",
            method = RequestMethod.POST
    )
    public ResponseEntity<Object> compile_java(@RequestPart(value = "output", required = true) MultipartFile output,
                                          @RequestPart(value = "sourceCode", required = true) MultipartFile sourceCode,
                                          @RequestParam(value = "inputFile", required = false) MultipartFile inputFile,
                                          @RequestParam(value = "timeLimit", required = true) int timeLimit,
                                          @RequestParam(value = "memoryLimit", required = true) int memoryLimit
    ) throws Exception {
        return compiler(output, sourceCode, inputFile, timeLimit, memoryLimit, Langage.Java);
    }
Enter fullscreen mode Exit fullscreen mode

What type of response should the user expect?

Well, let's take a minute on this point. If you are doing competitive programming on platforms like Codeforces, Leetcode, or others you can see that there is 6 types of verdict (Accepted, Wrong answer, Compilation error, Runtime error, Out of memory error, and Time limit exceeded).

Compiling the source code inside a docker container

The idea here is to take the source code provided by the user and create a docker image depending on the language then run a container of this image to compile and execute the source code. Depending on the returning code of the container we can decide the verdict that we discussed before, but we need to make sure that the container doesn't exceed the time limit to avoid infinite executions, neither the memory limit (to avoid malicious code).

// Compile method
    private ResponseEntity<Object> compiler(
            MultipartFile output,
            MultipartFile sourceCode,
            MultipartFile inputFile,
            int timeLimit,
            int memoryLimit,
            Langage langage
    ) throws Exception {

        String folder = "utility";
        String file = "main";
        if(langage == Langage.C) {
            folder += "_c";
            file += ".c";
        } else if(langage == Langage.Java) {
            file += ".java";
        } else if(langage == Langage.Cpp) {
            folder += "_cpp";
            file += ".cpp";
        } else {
            folder += "_py";
            file += ".py";
        }

        if(memoryLimit < 0 || memoryLimit > 1000)
            return ResponseEntity
                    .badRequest()
                    .body("Error memoryLimit must be between 0Mb and 1000Mb");

        if(timeLimit < 0 || timeLimit > 15)
            return ResponseEntity
                    .badRequest()
                    .body("Error timeLimit must be between 0 Sec and 15 Sec");

        LocalDateTime date = LocalDateTime.now();

        createEntrypointFile(sourceCode, inputFile, timeLimit, memoryLimit, langage);

        logger.info("entrypoint.sh file has been created");

        saveUploadedFiles(sourceCode, folder + "/" + file);
        saveUploadedFiles(output, folder + "/" + output.getOriginalFilename());
        if(inputFile != null)
            saveUploadedFiles(inputFile, folder + "/" + inputFile.getOriginalFilename());
        logger.info("Files have been uploaded");

        String imageName = "compile" + new Date().getTime();

        logger.info("Building the docker image");
        String[] dockerCommand = new String[] {"docker", "image", "build", folder, "-t", imageName};
        ProcessBuilder probuilder = new ProcessBuilder(dockerCommand);
        Process process = probuilder.start();
        int status = process.waitFor();
        if(status == 0)
            logger.info("Docker image has been built");
        else
            logger.info("Error while building image");

        logger.info("Running the container");
        dockerCommand = new String[] {"docker", "run", "--rm", imageName};
        probuilder = new ProcessBuilder(dockerCommand);
        process = probuilder.start();
        status = process.waitFor();
        logger.info("End of the execution of the container");

        BufferedReader outputReader = new BufferedReader(new InputStreamReader(output.getInputStream()));
        StringBuilder outputBuilder = new StringBuilder();
        BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
        StringBuilder builder = new StringBuilder();

        boolean ans = runCode(outputReader, outputBuilder, reader, builder);

        String result = builder.toString();

        // delete files
        deleteFile(folder, file);
        new File(folder + "/" + output.getOriginalFilename()).delete();
        if(inputFile != null)
            new File(folder + "/" + inputFile.getOriginalFilename()).delete();
        logger.info("files have been deleted");

        String statusResponse = statusResponse(status, ans);

        logger.info("Status response is " + statusResponse);

        return ResponseEntity
                .status(HttpStatus.OK)
                .body(new Response(builder.toString(), outputBuilder.toString(), statusResponse, ans, date));
    }
Enter fullscreen mode Exit fullscreen mode

The entrypoint of the container

To configure the entrypoint of the docker image, we create during the execution an entrypoint file that contains the time limit, memory limit and the execution command.

This is an example of how to generate the entrypoint.sh file for java execution :

// create Java entrypoint.sh file
    private void createJavaEntrypointFile(String fileName, int timeLimit, int memoryLimit, MultipartFile inputFile) {
        String executionCommand = inputFile == null
                ? "timeout --signal=SIGTERM " + timeLimit + " java " + fileName.substring(0,fileName.length() - 5) + "\n"
                : "timeout --signal=SIGTERM " + timeLimit + " java " + fileName.substring(0,fileName.length() - 5) + " < " + inputFile.getOriginalFilename() + "\n";
        String content = "#!/usr/bin/env bash\n" +
                "mv main.java " + fileName+ "\n" +
                "javac " + fileName + "\n" +
                "ret=$?\n" +
                "if [ $ret -ne 0 ]\n" +
                "then\n" +
                "  exit 2\n" +
                "fi\n" +
                "ulimit -s " + memoryLimit + "\n" +
                 executionCommand +
                "exit $?\n";
        OutputStream os = null;
        try {
            os = new FileOutputStream(new File("utility/entrypoint.sh"));
            os.write(content.getBytes(), 0, content.length());
        } catch (IOException e) {
            e.printStackTrace();
        }finally{
            try {
                os.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
Enter fullscreen mode Exit fullscreen mode

Talk is cheap, show me the code 😃

This was just an overview, instead of explaining how to code all this, I suggest you to visit the link of my Github repository https://github.com/zakariamaaraki/RemoteCodeCompiler

If you have any questions please feel free to ask them !

Discussion (0)

pic
Editor guide