Stream Fundamentals

What is a Stream?

Speedment is all about Java 8 Streams, that allows you to process data in a declarative way similar to SQL statements. So if you are not familiar with the concept of Streams, I encourage you to read this chapter carefully. If you consider yourself a Stream expert, feel free to skip directly to the next chapter.

A Java 8 Stream is an interface with implementations that supports a functional style operation on streams of elements. The entire Java Collection framework was retrofitted with Stream support in Java 8.

Streams can be used to express a kind of “recipe” style of operations, allowing us to compose a number of function and only when the Stream is started, the functions are applied to the elements in the Stream. This means that Streams can be very efficient and general. A Stream recipe says what to do but generally not how to do it which is good from an abstraction point of view.

Consider the following simple example:

    Stream.of(                           // Creates a Stream with the given content
        "Zlatan", 
        "Tim", 
        "Bo",
        "George",
        "Adam",
        "Oscar"
    )
        .filter(n -> n.length() > 2)     // Retains only those Strings that are longer than 2 characters (i.e. "Bo" is dropped)
        .sorted()                        // Sorts the remaining Strings in natural order
        .collect(Collectors.toList());   // Collects the remaining sorted Strings in a List

Start by creating a Stream using the statement Stream.of(). Note that nothing happens with the Stream at this point. This yields a Stream which can be used to further build a “recipe” around. By adding a filter only Strings that are longer than 2 characters will be included. Again, the Stream is not started, this only tells the Stream that when it starts, the Strings should be filtered. Next, a sorted() operation is added to the Stream recipe. This means that when the Stream is started, all Strings that passes the filter shall be sorted in natural order. Again, nothing is flowing through the Stream, the Stream recipe (the stream recipe can more formally be called a stream pipeline) now holds yet an operation. The last operation to be added is collect.

This operation is different to all the previous operations in the way that it is a Terminal operation. Whenever a Terminal operation is applied to a Stream, the Stream cannot accept additional operations to its pipeline. It also means that the Stream is started.

It shall be noted that elements in a Stream are pulled by the Terminal Operation (i.e. the collect operation) and not pushed by the stream source. So, Collect will ask for the first element and that request will traverse up to the stream source that will provide the first element “Zlatan”. The filter operation will check if the length of “Zlatan” is greater than two (which it is) and will then propagate “Zlatan” to the sorted operation. Because the sorted operation needs to see all strings before it can decide on its output order, it will ask the stream source for all its remaining elements which, via the filter, is sent down the stream. Once all strings are received by the sorted operator, it will sort the strings and then output its first element (i.e. “Adam”) to the collect operation. The result of the entire stream statement will thus be:

"Adam", "George", "Oscar", "Tim", "Zlatan"

Streams with Speedment

With Speedment, it is possible to use exactly the same semantics as for Java 8 streams. Instead of Strings as shown in the example above, database rows can be streamed. This way, database tables can be viewed as pure Java 8 Streams as shown hereunder:

    users.stream()                       // Creates a Stream with users from a database 
        .map(u -> u.getName())           // Extract the name (a String) from a user
        .filter(n -> n.length() > 2)     // Retains only those Strings that are longer than 2 characters (i.e. "Bo" is dropped)
        .sorted()                        // Sorts the remaining Strings in natural order
        .collect(Collectors.toList());   // Collects the remaining sorted Strings in a List

Because a Java 8 Stream is an interface, Speedment can select from a variety of different implementations of a Stream depending on the pipeline that is used and other factors.

Speedment Stream Order

The order in which elements are produced by the stream is unspecified and may change from one invocation to another. Because of that, it is an error to assume any particular order. Use the intermediate operation sorted(Comparator) if you need a certain element order.

Intermediate Operations

An Intermediate Operation is an operation that allows further operations to be added to a Stream. For example, filter is an Intermediate Operation because additional operations can be added to a Stream pipeline after filter has been applied to the Stream.

Common Operations

The following Intermediate Operations can be accepted by a Stream:

Stream Property Operations

There are also a number of Intermediate Operations that controls the properties of the Stream and has no effect on its actual content. These are:

Map to Primitive Operations

There are also some Intermediate Operations that maps a Stream to one of the special primitive stream types; IntStrem, LongStream and DoubleStream:

Primitive streams provides better performance in many cases but can only handle streams of: int, long and double.

Primitive Operations

Primitive streams (like IntStream and LongStream) provide similar functionality as ordinary streams but usually the parameter count and types differ so that primitive streams can accept more optimized function variants. Here is a table of some additional Intermediate Operations that primitive Streams can take:

Java 9 Operations

Two new Intermediate Operations were introduced in Java 9. Because these methods were added to the Stream interface with default implementations, these methods can be used by any Stream implementation written in either Java 8 or Java 9.

Please revise the complete Stream JavaDoc for more information. Here are some examples of streams with Intermediate Operations:

Intermediate Operations Examples

Here is a list with examples for many of the Intermediate Operations. In the examples below, lambdas are used but many times, the lambdas could be replaces by method references (e.g. the lambda () -> new StringBuilder can be replaced by a method reference StringBuilder::new). The source code for the examples with Intermediate Operations below can be found here on GitHub

filter

    Stream.of("B", "A", "C" , "B")
        .filter(s -> s.equals("B"))

returns a Stream with the elements “B” and “B” because only elements that are equal to “B” will pass the filter operation.

map

    Stream.of("B", "A", "C" , "B")
        .map(s -> s.toLowerCase())

is a Stream with the elements “b”, “a”, “c” and “b” because each element will be mapped (converted) to its lower case representation.

distinct

    Stream.of("B", "A", "C" , "B")
        .distinct()

is a Stream with the elements “B”, “A” and “C” because only unique elements will pass the distinct operation.

sorted

    Stream.of("B", "A", "C" , "B")
        .sorted()

returns a Stream with the elements “A”, “B”, “B” and “C” because the sort operation will sort all elements in the stream in natural order.

    Stream.of("B", "A", "C" , "B")
        .sorted(Comparator.reverseOrder())

is a Stream with the elements “C”, “B”, “B” and “A” because the sort operation will sort all elements in the stream according to the provided Comparator (in reversed natural order).

limit

    Stream.of("B", "A", "C" , "B")
        .limit(2)

is a Stream with the elements “B” and “A” because after the two first elements the rest of the elements will be discarded.

skip

    Stream.of("B", "A", "C" , "B")
        .skip(1)

is a Stream with the elements “A”, “C” and “B” because the first element in the stream will be skipped.

flatMap

    Stream.of(
        Stream.of("B", "A"),
        Stream.of("C", "B")
    )
        .flatMap(Function.identity())
        .forEachOrdered(System.out::println);

returns a Stream with the elements “B”, “A”, “C” and “B” because the two streams (that each contain two elements) are “flattened” to a single Stream with four elements.

    Stream.of(
        Arrays.asList("B", "A"),
        Arrays.asList("C", "B")
    )
        .flatMap(l -> l.stream())

returns a Stream with the elements “B”, “A”, “C” and “B” because the two lists (that each contain two elements) are “flattened” to a single Stream with four elements. The lists are converted to sub-streams using the List::stream mapper method.

peek

    Stream.of("B", "A", "C" , "B")
        .peek(System.out::print)

is a Stream with the elements “B”, “A”, “C” and “B” but, when consumed in its entirety, will print out the text “BACB” as a side effect. Note that side effect usage in stream are discouraged. Use this operation for debug only.

parallel

    Stream.of("B", "A", "C" , "B")
        .parallel()

is a Stream with the elements “B”, “A”, “C” and “B” but, when consumed, elements in the Stream may be propagated through the pipeline using different Threads. By default, parallel streams are executed on the default ForkJoinPool.

sequential

    Stream.of("B", "A", "C" , "B")
        .parallel()
        .sequential()

is a Stream with the elements “B”, “A”, “C” and “B” that is not parallel.

unordered

    Stream.of("B", "A", "C" , "B")
        .unordered()

is a Stream with the given elements but not necessary in any particular order. So when consumed, elements might be encountered in any order, for example in the order “C”, “B”, “B”, “A”. Note that unordered is just a relaxation of the stream requirements. Unordered streams can retain their original element order or elements can appear in any other order.

onClose

    Stream.of("B", "A", "C", "B")
        .onClose( () -> System.out.println("The Stream was closed") );

is a Stream with the elements “B”, “A”, “C” and “B” but, when closed, will print out the text “The Stream was closed”.

mapToInt

    Stream.of("B", "A", "C" , "B")
        .mapToInt(s -> s.hashCode())

is an IntStream with the int elements 66, 65, 67 and 66. (A is 65, B is 66 and so on)

mapToLong

    Stream.of("B", "A", "C", "B")
        .mapToLong(s -> s.hashCode() * 1_000_000_000_000l)

is a LongStream with the long elements 66000000000000, 65000000000000, 67000000000000 and 66000000000000.

mapToDouble

    Stream.of("B", "A", "C", "B")
        .mapToDouble(s -> s.hashCode() / 10.0)

is a DoubleStream with the double elements 6.6, 6.5, 6.7 and 6.6.

flatMapToInt

    Stream.of(
        IntStream.of(1, 2),
        IntStream.of(3, 4)
    )
        .flatMapToInt(s -> s.map(i -> i + 1))

is an IntStream with the int elements 2, 3, 4 and 5 because the two IntStreams where flattened to one stream whereby 1 was added to each element.

flatMapToLong

    Stream.of(
        LongStream.of(1, 2),
        LongStream.of(3, 4)
    )
        .flatMapToLong(s -> s.map(i -> i + 1))

is a LongStream with the long elements 2, 3, 4 and 5 because the two LongStreams where flattened to one stream whereby 1 was added to each element.

flatMapToDouble

    Stream.of(
        LongStream.of(1.0, 2.0),
        LongStream.of(3.0, 4.0)
    )
        .flatMapToDouble(s -> s.map(i -> i + 1))

is a DoubleStream with the double elements 2.0, 3.0, 4.0 and 5.0 because the two DoubleStreams where flattened to one stream whereby 1 was added to each element.

boxed

    IntStream.of(1, 2, 3, 4)
        .boxed()

is a Stream with the Integer elements 1, 2, 3 and 4 because the original int elements were boxed to their corresponding Integer elements.

asLongStream

     IntStream.of(1, 2, 3, 4)
        .asLongStream()

is a LongStream with the long elements 1, 2, 3 and 4 because the original int elements were converted to long elements.

asDoubleStream

    IntStream.of(1, 2, 3, 4)
        .asDoubleStream()

is a DoubleStream with the double elements 1.0, 2.0, 3.0 and 4.0 because the original int elements were converted to double elements.

takeWhile (Java 9 only)

    Stream.of("B", "A", "C", "B")
        .takeWhile(s -> "B".compareTo(s) >= 0)

is a Stream with the elements “B” and “A” because when “C” in encountered in the stream, that element and all following are dropped.

dropWhile (Java 9 only)

    Stream.of("B", "A", "C", "B")
        .dropWhile(s -> "B".compareTo(s) >= 0)

is a Stream with the elements “C” and “B” because elements are dropped from the stream but when “C” in encountered, subsequent elements are not dropped.

This completes the example list of Intermediate Operation examples.

Terminal Operations

A Terminal Operations starts the Stream and returns a result that depends on the Stream pipeline and content. For example, collect is a Terminal Operation because additional operation cannot be added to a Stream pipeline after collect has been called.

Common Operations

Here are some common examples of Terminal Operations that can be accepted by a Stream:

Less Common Operations

Here is a list of other Terminal Operations that are a bit less commonly used by at least some programmers:

Primitive Stream Operations

Primitive streams (like IntStream and LongStream) provide similar functionality as ordinary streams but usually the parameter count and types differ so that primitive streams can accept more optimized function variants. Here is a list of additional Terminal Operations that are available for primitive Streams:

Please revise the complete Stream JavaDoc for more information.

Terminal Operations Examples

Here is a list with examples for many of the Terminal Operations. The source code for the examples below with Terminal Operations can be found here on GitHub

forEach

     Stream.of("B", "A", "C" , "B")
        .forEach(System.out::print);

might output “CBBA”. However, it has to be said that most stream implementation actually would output “BACB” but there is no guarantee of a particular order using forEach.

forEachOrdered

     Stream.of("B", "A", "C" , "B")
        .forEachOrdered(System.out::print);

outputs “BACB” (always as opposed to forEach)

collect

     Stream.of("B", "A", "C" , "B")
        .collect(Collectors.toList());

Returns a List<String> equal to [“B”, “A”, “C”, “B”]

     Stream.of("B", "A", "C" , "B")
        .collect(Collectors.toSet());

Returns a Set<String> equal to [“A”, “B”, “C”]

    Stream.of("I", "am", "a", "stream")
        .collect(Collectors.toMap(
            s -> s.toLowerCase(), // Key extractor
            s -> s.length())      // Value extractor
        )

Returns a Map<String, Integer> equal to {a=1, stream=6, i=1, am=2}. Thus, the Map contains a mapping from a word (key) to how many characters that word has (value).

min

     Stream.of("B", "A", "C" , "B")
        .min(String::compareTo);

returns Optional[A] because “A” is the smallest element in the stream.

    Stream.<String>empty()
        .min(String::compareTo);

returns Optional.empty because there is no min value because the stream is empty.

max

     Stream.of("B", "A", "C" , "B")
        .max(String::compareTo);

returns Optional[C] because “C” is the largest element in the stream.

    Stream.<String>empty()
        .max(String::compareTo);

returns Optional.empty because there is no max value because the stream is empty.

count

     Stream.of("B", "A", "C" , "B")
        .count();

returns 4 because there are four elements in the stream.

    Stream.empty()
        .count();

returns 0 because there are no elements in an empty stream.

anyMatch

    Stream.of("B", "A", "C", "B")
        .anyMatch("A"::equals);

returns true because there is an “A” element in the stream.

    Stream.of("B", "A", "C", "B")
        .anyMatch("Z"::equals);

returns false because there are no “Z” elements in the stream.

noneMatch

    Stream.of("B", "A", "C", "B")
        .noneMatch("A"::equals);

returns false because there is an “A” element in the stream.

    Stream.of("B", "A", "C", "B")
        .noneMatch("Z"::equals);

returns true because there are no “Z” elements in the stream.

findFirst

    Stream.of("B", "A", "C", "B")
        .findFirst();

returns Optional[B] because “B” is the first element in the stream.

    Stream.<String>empty()
        .findFirst();

returns Optional.empty because the stream is empty.

findAny

    Stream.of("B", "A", "C", "B")
        .findAny();

might return Optional[C] or any other element in the stream.

    Stream.<String>empty()
        .findAny();

returns Optional.empty because the stream is empty.

toArray

    Stream.of("B", "A", "C", "B")
        .toArray();

Returns an array containing [B, A, C, B] the array being created automatically by the toArray operator.

    Stream.of("B", "A", "C", "B")
        .toArray(String[]::new)

Returns an array containing [B, A, C, B] that will be created by the provided constructor, for example using the equivalent to new String[4].

collect with 3 Parameters

            Stream.of("B", "A", "C", "B")
                                .collect(
                    () -> new StringBuilder(),
                    (sb0, sb1) -> sb0.append(sb1),
                    (sb0, sb1) -> sb0.append(sb1)
                )

Returns a StringBuilder containing “BACB” that will be created by the provided supplier and then built up by the append lambdas.

reduce

    Stream.of(1, 2, 3, 4)
        .reduce((a, b) -> a + b)

Returns the value of Optional[10] because 10 is the sum of all Integer elements in the stream. If the stream is empty, Optional.empty() is returned.

    Stream.of(1, 2, 3, 4)
        .reduce(100, (a, b) -> a + b)

Returns the value of 110 since all the Integer elements in the Stream are added to the Integer 100. If the Stream is empty, 100 is returned.

    Stream.of(1, 2, 3, 4)
        .parallel()
        .reduce(
            0, 
            (a, b) -> a + b,
            (a, b) -> a + b
        )

Returns the value of 10 since this example simply adds all the Integer elements in the Stream beginning with 0. The Stream can be executed i parallel whereby the last lambda will be used to combine results from each thread. If the Stream is empty, 0 is returned.

iterator

    Iterator<String> iterator
        = Stream.of("B", "A", "C", "B")
            .iterator();

Creates a new Iterator over all the elements in the Stream.

spliterator

    Spliterator<String> spliterator
        = Stream.of("B", "A", "C", "B")
            .spliterator();

Creates a new Spliterator over all the elements in the Stream.

sum

    IntStream.of(1, 2, 3, 4)
        .sum()

Returns 10 because 10 is the sum of all elements in the Stream.

average

    IntStream.of(1, 2, 3, 4)
        .average()

Returns OptionalDouble[2.5] because 2.5 is the average of all elements in the Stream. If the Stream is empty, OptionalDouble.empty() is returned.

summaryStatistics

    IntStream.of(1, 2, 3, 4)
        .summaryStatistics()

Returns IntSummaryStatistics{count=4, sum=10, min=1, average=2.500000, max=4}.

If the stream is empty, IntSummaryStatistics{count=0, sum=0, min=2147483647, average=0.000000, max=-2147483648} is returned (max is initially set to Integer.MIN_VALUE which is -2147483648 and min is set to Integer.MAX_VALUE which is 2147483648).

Other Operations

There are also a small number of other operations that are neither a Intermediate Operation nor a Terminal Operation as shown in the table below:

Please revise the complete Stream JavaDoc for more information.

Other Operations Examples

Here is a list with example of other operations. The source code for the examples below with other operations can be found here on GitHub

isParallel

    Stream.of("B", "A", "C", "B")
        .parallel()
        .isParallel()

Returns true because the Stream is parallel.

    Stream.of("B", "A", "C", "B")
        .sequential()
        .isParallel()

Returns false because the Stream is not parallel.

close

    Stream<String> stream = Stream.of("B", "A", "C", "B");
    stream.forEachOrdered(System.out::println);
    stream.close();

Prints all elements in the stream and then closes the stream. Some streams (e.g. streams from files) need to be closed to release their resources. Use the try-with-resource patterns if the stream must be closed:

    try (Stream<String> s = Stream.of("B", "A", "C", "B")) {
        s.forEachOrdered(System.out::println);
    }

Examples

The examples below operates on entities of type User. Note that the only thing that needs to be provided is the id and all the other fields will be derived from the this field. This is good for tests and examples but of course not so useful in real applications. The User class looks like this:

    static class User {

        private static final List<String> CARS = Arrays.asList("Toyota", "Volvo", "Tesla", "Fiat", "Ford");

        private final int id;

        public User(int id) {
            this.id = id;
        }

        public int getId() {
            return id;
        }

        public String getName() {
            return "Name" + id;
        }

        public String getPassword() {
            return "PW" + (id ^ 0x7F93A27F);
        }

        public String getFavoriteCar() {
            return CARS.get(id % CARS.size());
        }

        public int getBornYear() {
            return 1950 + id % 50;
        }

        @Override
        public String toString() {
            return String.format(
                "{id=%d, name=%s, password=%s, favoriteCar=%s, bornYear=%d}",
                getId(),
                getName(),
                getPassword(),
                getFavoriteCar(),
                getBornYear()
            );
        }

        @Override
        public boolean equals(Object obj) {
            if (!(obj instanceof User)) {
                return false;
            }
            User that = (User) obj;
            return this.id == that.id;
        }

        @Override
        public int hashCode() {
            return id;
        }

    }

As previously stated, users are really not real users but instead they are synthetically generated from the user id. Because the id defines all other fields, only the id field needs to be used in the equals and hashCode methods.

The first users will thus be:

There is also a UserManager that provides a static stream method that will return a Stream<User> that contains 1000 elements (with user ids in the range 0 to 999). The UserManager class is shown hereunder:

    static class UserManager {
        static Stream<User> stream() {
            return IntStream.range(0, 1000)
                .mapToObj(User::new);
        }
    }

Note how the stream method creates an IntStream with elements from 0 to 999 and then maps each int element to a User object using the User constructor that takes an int as an argument.

Count the Number of Ford Likers

The example below counts the number of users that like Ford:

    long count = UserManager.stream()
        .filter(u -> "Ford".equals(u.getFavoriteCar()))
        .count();

        System.out.format("There are %d users that supports Ford %n", count);

The code above will produce:

There are 200 users that supports Ford 

Calculate Average Age

The following example calculates the average age of the users that like Tesla assuming the current year is 2017:

    OptionalDouble avg = UserManager.stream()
        .filter(u -> "Tesla".equals(u.getFavoriteCar()))
        .mapToInt(u -> 2017 - u.getBornYear()) // Calculates the age
        .average();

    if (avg.isPresent()) {
        System.out.format("The average age of Tesla likers are %d years %n", avg.getAsDouble());
    } else {
        System.out.format("There are no Tesla lovers");
    }

The code above will produce:

The average age of Tesla likers are 42.500000 years

Find the Youngest Volvo Digger

The next example sets out to locate the youngest Volvo digger. The solution imposed below sorts all users in bornYear order and then picks the first one. Is there another solution without sort?

Comparator<User> comparator = Comparator.comparing(User::getBornYear).reversed();
        
    Optional<User> youngest = UserManager.stream()
        .filter(u -> "Volvo".equals(u.getFavoriteCar()))
        .sorted(comparator)
        .findFirst();

    youngest.ifPresent(u
        -> System.out.println("Found the youngest Volvo digger which is :" + u.toString())
    );

This will produce the following output:

Found the youngest Volvo digger which is :{id=46, name=Name46, password=PW782496767, favoriteCar=Volvo, bornYear=1996}

Collect a Stream in a List

The following example collects all users that love Fiat in a List:

        List<User> fiatLovers = UserManager.stream()
            .filter(u -> "Fiat".equals(u.getFavoriteCar()))
            .collect(Collectors.toList());

        System.out.format("There are %d fiat lovers %n", fiatLovers.size());

The code above will produce:

There are 200 fiat lovers

Element Flow

The example below examines the flow of elements and the different operations in the stream’s pipeline. A Stream with five names is created and a filter is used to find those having a name that starts with the letter “A”. A sort operation is also applied to the remaining names and then lastly the names are mapped to lower case. Lastly the remaining elements are printed. Print statements are used in between operations to enable observation of the separate operation:

    Stream.of("Bert", "Alice", "Charlie", "Assian", "Adam")
        .filter(s -> {
            String required = "A";
            boolean result = s.startsWith(required);
            System.out.format("filter        : \"%s\".startsWith(\"%s\") is %s (%s) %n", s, required, result, result ? "retained" : "dropped");
            return result;
        })
        .sorted((s1, s2) -> {
            int result = s1.compareTo(s2);
            System.out.format("sort          : compare(%s, %s) is %d (%s)%n", s1, s2, result, result < 0 ? "not swapped" : "swapped");
            return result;
        })
        .map(s -> {
            String result = s.toLowerCase();
            System.out.format("map           : %s -> %s %n", s, result);
            return result;
        })
        .forEachOrdered(s
            -> System.out.println("forEachOrdered: " + s)
        );

This will print:

filter        : "Bert".startsWith("A") is false (dropped) 
filter        : "Alice".startsWith("A") is true (retained) 
filter        : "Charlie".startsWith("A") is false (dropped) 
filter        : "Assian".startsWith("A") is true (retained) 
filter        : "Adam".startsWith("A") is true (retained) 
sort          : compare(Assian, Alice) is 7 (swapped)
sort          : compare(Adam, Assian) is -15 (not swapped)
sort          : compare(Adam, Assian) is -15 (not swapped)
sort          : compare(Adam, Alice) is -8 (not swapped)
map           : Adam -> adam 
forEachOrdered: adam
map           : Alice -> alice 
forEachOrdered: alice
map           : Assian -> assian 
forEachOrdered: assian

So, in the end, the stream delivered the elements “adam”, “alice” and “assian” as expected. Note how sort needs to retrieve all the element via the filter stage before it can emit result to the next stage. On the contrary, the last steps are executed in pipeline order because both map and forEachOrdered can process a stream element one at a time.

Questions and Discussion

If you have any question, don’t hesitate to reach out to the Speedment developers on Gitter.