Example : Bulk loading files

Loading a large number of files into memory is something that can often be greatly speeded up by making the task asynchronous. To do this with SimpleReact

Loading files asynchronously

In the example below, we create a list of Suppliers, each of which will Supply a file loaded for disk.

 public void loadingFilesFromDisk(List<File> toLoad){

  FileReader reader = new FileReader();
	
	final List<Supplier<LoadedData>> suppliers = toLoad.stream()
                                                               .<Supplier<LoadedData>>map( file ->   { return () -> reader.readFileFromDisk(file); })
                                                                .collect(Collectors.toList());
	
	final List<LoadedData> loadedData = new SimpleReact().react(suppliers).block();

The same code not using Java 8 Streams would look like this

 FileReader reader = new FileReader();
	
	
	final List<Supplier<LoadedData>> suppliers = new ArrayList<>();
	for (final File reference : toLoad) {

		suppliers.add(()  ->{
			
				
				return reader.readFileFromDisk(reference);
			
		});

	}
	
	final List<LoadedData> loadedData = new SimpleReact().react(suppliers).block();

Loading files asynchronously while the current thread is also not blocked

In the previous code example, the current thread was blocked while data was loaded from disk. Instead of calling block immediately we can, instead, carry on processing and only call block when we are ready for the results.

e.g.

final List<Supplier<LoadedData>> suppliers = toLoad.stream().<Supplier<LoadedData>>map( file ->   { return () -> reader.readFileFromDisk(file); }).collect(Collectors.toList());
	

	Stage<LoadedData>  stage = new SimpleReact().react(suppliers);
	
	for(int i=0;i<100000;i++){
		System.out.println("Current thread can continue working while data loads");
	}
	
	final List<LoadedData> loadedData =stage.block();

Processing loaded files without blocking the current thread at all

We don't need to call block to access the data for additional processing. We can let the reactive dataflow continue by defining more stages. For example the next stage might be to process the loaded data in some manner. We could then filter out data that didn't meet some important criteria, before saving data that did back to disk.

e.g.

 FileReader reader = new FileReader();
	
	final List<Supplier<LoadedData>> suppliers = toLoad.stream().<Supplier<LoadedData>>map( file ->   { return () -> reader.readFileFromDisk(file); }).collect(Collectors.toList());
	


	 
	new SimpleReact().react(suppliers)
					.then(loadedData -> processLoadedData(loadedData))
					.filter(processedData -> meetsCriteria(processedData))
					.then(filteredData -> writeToDisk(filteredData));