Iterator Guidelines

We'd like all cloud API clients written in Go to be as consistent with each other as possible. Since Go has no standard iterator pattern, there is a risk that different clients will express iteration in different ways. So we've come up with some guidelines.

Most iterators will result from the standard Google APIs List method with the pagination pattern: each call to a List method for a resource returns a sequence (“page”) of resource items (e.g. Books) along with a “next-page token” that can be passed to the List method to retrieve the next page.

Each List method will result in an iterator (which we call a List iterator) with two methods, one for individual items and one for pages. Iterators may also arise from other sources, like streaming RPCs or Cloud PubSub message subscriptions. These may or may not support page-by-page iteration.

Examples

Here is what iterators written according to these guidelines will look like to users. Here is the List iterator for the Book resource in the library example:

it := client.Books(ctx, shelfName)
for {
	book, err := it.Next()
	if err == library.Done {
		break
	}
	if err != nil {
		return err
	}
	process(book)
}

Here's the same code using a switch:

it := client.Books(ctx, shelfName)
loop: for {
	book, err := it.Next()
	switch err {
	case nil: process(book)
	case library.Done: break loop
	default: return err
}

Here is what iteration by pages looks like:

it := client.Books(ctx, shelfName)
for {
	books, err := it.NextPage()
	if err == library.Done {
		break
	}
	if err != nil {
		return err
	}
	for _, b := range books {
		process(b)
	}
}

Here we retrieve the first page of 25 (or fewer) books and display the page and the next-page token:

it := client.Books(ctx, shelfName)
it.SetPageSize(25)
books, err := it.NextPage()
if err == nil {
	display(books, it.NextPageToken())
}

When the next-page token is handed back to us later (possibly in another process), we can get the next page:

it := client.Books(ctx, shelfName)
it.SetPageSize(25)
it.SetPageToken(token)
books, err := it.NextPage()
if err == nil {
	display(books, it.NextPageToken())
}

Guidelines

1. The name of an iterator type should end in Iterator.
2. The name of a List iterator type should be ResourceIterator. E.g. BookIterator.
3. The name of the method creating a List iterator should be the plural of the resource, E.g. Books (not ListBooks). For other kinds of iterators, the name of the creating method should be a plural noun, unless there is a better choice.
4. The first argument to the iterator-creating method should be a context, unless the method and iterator make no RPCs. The iterator will use that context.
5. If iteration makes any RPCs, the iterator-creating method should accept call options in a final ... parameter, and store them for subsequent RPCs.
6. In most cases, the iterator-creating method will not return an error along with the iterator, but it is permitted to do so. (It won't need to return an error because it will just create the iterator, leaving the initial work to the first call to Next.)
7. An iterator over values of type T will have a method called Next that returns (T, error).
8. Following standard Go convention, if Next’s second return value is non-nil, then the first must be the zero value for T.
9. Next will typically have no arguments, but it may in rare cases. None of the arguments should be a context, because Next should use the context passed when the iterator was created.
10. A special error value returned by Next signals the successful end of the iteration. This sentinel value will be named Done (unless that results in a conflict) and will be declared as a variable in the same package as the iterator.
11. After Next returns Done, all subsequent calls to it will return Done.
12. If feasible, the user should be able to continue calling Next even if it returns an error that is not Done. If that is not feasible, it should be so documented.
13. List iterators also have a NextPage method that returns ([]Resource, error).
14. List iterators have a method NextPageToken, returning the next-page token.
15. List iterators should have a SetPageToken(string) method which sets the page token for the next List RPC.
16. You don't need to explicitly use page tokens if you just want consecutive pages: repeated calls to NextPage will accomplish that. You'll get the same behavior if you call SetPageToken(NextPageToken()) between calls to NextPage.
17. Again following standard Go convention, if NextPage’s second return value is non-nil, then the first must be nil.
18. NextPage may return (r, nil) where len(r) == 0. In other words, there can be an empty page in the middle of the iteration.
19. List iterators should have a SetPageSize(int32) method which sets the page size for all subsequent List RPCs from the current iterator. In the usual case, the page size is a maximum. Some iterators may choose, for user convenience, to return exactly that number of items (if they are available).
20. List iterators are not required to be “smart” about interleaved calls to Next and NextPage. We expect users to call one consistently for the life of an iterator.
21. An iterator may require clean-up work to happen after it completes, or if the user abandons it before reaching the end. Such an iterator should define a method named Close, with no arguments. Close may, but need not, have a single error return value. There should not be a Close method unless it is necessary, and it should not have an error return unless it might result in an error.
22. If feasible, Close should support being called multiple times. If it can't be, that fact should be documented.
23. Next and NextPage are not required to be thread-safe, and Close, Next and NextPage are not required to be concurrently callable.
24. Iterators may have fields and methods other than those described here if necessary.