net/http: expose matched pattern in Request

[sillydong]

Proposal Details

Go 1.22 has provided http.ServeMux with supporting of pattern match for requests. While I'm using it, I tried to get the matched pattern of that request. With help of *ServeMux.Handler, I can successfully get matched handler and pattern, but later when I run debug of the code, I found in *http.Request itself, there is a private pat which is exactly matched pattern info. I suggest we export the pat for public access or provide some function to allow user to get the matched pattern, so that we can avoid much logic to use *ServeMux.Handler to parse the request again. This reduces huge resource usage in high load services.

The matched pattern info is usually very useful when we do analysis for requests. We can group requests by their pattern and make the analysis more generic. This is a very high frequency usage scenario which makes reducing the resource usage very worthy.

sample code:

func TestServeMux(t *testing.T) {
	m := http.NewServeMux()
	m.Handle("GET /{name}", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		t.Log(r.PathValue("name"))
	}))
	http.ListenAndServe(":8999", m)
}

When we do request GET http://127.0.0.1:8999/aloha, we can see such info in debug console.

Update 2024/03/28:
As we talked in comments, @jba 's suggestion is a good solution #66405 (comment)

// Pattern returns the pattern that matched this request, if the request
// resulted from matching a pattern registered on a ServeMux.
// Otherwise, it returns the empty string.
func (*Request) Pattern() string

Update 2024/04/11:
After many discussions, the latest proposal can be found here #66405 (comment)
In short, add a Pattern field in http.Request to expose matched pattern info and can be set by third party routers.

Update 2024/05/13:
Final design: #66405 (comment)
The proposal is to add a Pattern string field to net/http.Request.
ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux.
In Go 1.21 compatibility mode the field is unset.
The field is otherwise left untouched. Third-party routers may set it if they wish.

[seankhliao]

see also #61551

[sillydong]

Similar to #61551 but instead of setting pattern info in Context, I suggest we export it directly in http.Request where it already has that info and the pattern is actually corresponding to request itself instead of any Context.

[AlexanderYastrebov]

Maybe the string pattern value could be made accessible via special name r.PathValue("*") (note also #64910)

[sillydong]

We definitely need a way to get pattern info in lower cost instead of using *ServeMux.Handler to parse request again.

For now there are three suggestions for getting that

• expose pat property in request directly or provide a function Pattern to get pattern string
• let servemux set matched pattern info in context, people get pattern from context by fixed key proposal: net/http: add PatternContextKey package variable to track routing #61551
• use a special name r.PathValue("*") to get pattern

[ianlancetaylor]

CC @neild @jba

[jba]

I wouldn't be opposed to adding:

// Pattern returns the pattern that matched this request, if the request
// resulted from matching a pattern registered on a ServeMux.
// Otherwise, it returns the empty string.
func (*Request) Pattern() string

[jba]

@sillydong, if you're OK with my suggestion above, please edit the top post to refer to my comment. You may also want to edit the issue title.

[DeedleFake]

What about custom mux implementations? Should they be able to store whatever pattern they used in the Request?

[neild]

Request.PathValue already conveys information from the mux, without any way for custom mux implementations to set it.

Perhaps there should be some way for custom muxes to set Request.Pattern and Request.PathValue, but I think that can be a separate proposal. Adding Pattern doesn't seem to make things any worse for custom muxes than they are now.

[seankhliao]

Request.PathValue already conveys information from the mux, without any way for custom mux implementations to set it.

it can be set with Request.SetPathValue

[neild]

it can be set with Request.SetPathValue

Today I learned!

In that case, I agree that there should be a corresponding SetPattern, or Pattern should just be a field of Request.

[jba]

I'm OK with a Pattern field.

[sillydong]

To expose the Pattern field, we have to change private pattern to public Pattern, also the fields in pattern like str,method,host,segments,loc should be public, and this makes type segment should also be public. This is making things complicate. I suggest we provide Pattern()string and SetPattern(string) functions. In SetPattern(string) we can call existing parsePattern to parse the parameter and set to http.Request.pat. This can be much easier and cleaner.
@jba @neild what do you think?

[gopherbot]

Change https://go.dev/cl/574997 mentions this issue: net/http: add Pattern and SetPattern in Request to return matched pattern info

[neild]

To expose the Pattern field, we have to change private pattern to public Pattern, also the fields in pattern like str,method,host,segments,loc should be public, and this makes type segment should also be public.

Or we could just have:

type Request {
  // Pattern is the [ServeMux] pattern that matched the request, if any.
  Pattern string
}

That seems pretty simple.

If we do have a SetPattern, then we need to consider how to handle a pattern that doesn't parse. What if the user is using a mux with patterns that don't correspond to the ones http.ServeMux uses? That seems likely; I'd expect the entire reason someone would use an alternate mux is to get different pattern matching features.

[sillydong]

@neild No. Pattern is actually a structured data. It contains segments and pattern string itself. Segments are used while getting path value. If we turn pattern into string, we lost everything.

I have created PR for Pattern()string and SetPattern(string)error, you can check it out.

[jba]

@sillydong I think you're missing @neild's point. Your code for SetPattern parses the pattern according to http.ServeMux's rules, but someone else's pattern syntax may be entirely different. Since we can't know how other patterns work, the best we can do is use a string.

[sillydong]

I think both you guys explained makes sense. But the problem is if we provide a function to set a string pattern, we have to create a new field to save that string info. That makes us not able to take advantage of Requst.pat. And also the Pattern function would have to return that string value instead of the real pattern string in Request.pat.str. This is kind of weird. I'm not quite sure about if this is a good way to handle the pattern info.

[jba]

We would be duplicating information, that's true. Request.Pattern would be set from Request.pat.str. If we were just supporting retrieving the pattern, I probably would have a Pattern() string method instead that returns Request.pat.str. But since we are allowing setting it, and setting it to an arbitrary string that might not be a valid pattern, having a separate field makes more sense.

[jba]

@sillydong My code was typed without testing, thanks for fixing it.
I agree that it should be the pattern of the innermost ServeMux. That's what Request.PathValue does, and it should be consistent.

Can I ask you once again to change the top post to reflect the current proposal, which is to have a Pattern field instead of a method.

Are there any unresolved objections to that?

[rsc]

FWIW, it is usually clearer to post the updated proposal in a new message, and then update the first comment to just say

"The latest proposal is <link to message>."

than to rewrite history. Rewriting history is confusing to anyone trying to get up to speed on the conversation.

[jba]

This comment describes the current proposal.

Add a Pattern field of type string to net/http.Request.
ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler.
The field is otherwise left untouched. Third-party routers may set it if they wish.

[sillydong]

Instead of checking r.pat, I took usage of the returned string of mux.findHandler(r). Easier and cleaner.

@rsc Proposal is updated and code change has been done. Please help push forward the progress. Thanks a lot. PR: golang.org/cl/574997

[rsc]

Have all remaining concerns about this proposal been addressed?

The proposal is to add a Pattern string field to net/http.Request.
ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux.
In Go 1.21 compatibility mode the field is unset.
The field is otherwise left untouched. Third-party routers may set it if they wish.

[sillydong]

In Go 1.21 compatibility mode the field is unset.

For this part, I think we can set matched pattern string to Pattern field because in Go 1.21 the mux is also able to run finding matched handler, it also returns matched pattern string.

[seankhliao]

Does accepting this mean declining #61551 ?
And will http.StripPrefix modify Request.Pattern ?

[jba]

Does accepting this mean declining #61551 ?

I think yes. I don't believe that proposal adds anything once we have Request.Pattern.

And will http.StripPrefix modify Request.Pattern ?

No. I don't even know what that would mean in some cases. For example, say the pattern is /{a}, the path is /foo, and the argument to StripPrefix is /f. That works today and changes the path to oo (it literally calls strings.TrimPrefix). What would it do to the pattern?

[seankhliao]

So there's #64909 to teach StripPrefix about path patterns.

I was wondering about a more limited form where the pattern is processed the same as Request.URL.Path and RawPath now, so a literal match would be updated
(request /a/b, handle /a/ stripprefix /a/, pattern /b),
but a pattern match wouldn't
(request /a/b, handle /{x}/, stripprefix ???, pattern /{x}/).

[sillydong]

In Go 1.21 compatibility mode the field is unset.

For this part, I think we can set matched pattern string to Pattern field because in Go 1.21 the mux is also able to run finding matched handler, it also returns matched pattern string.

@rsc @jba Hi, what do you think of this update? I think it's fair enough to also update the pattern for mux121 since it does have the pattern info from findHandler, we should not ignore it.

[sillydong]

Hi @rsc @jba , is there any update for this proposal? What do you think of also provide pattern info for Go 1.21 mux?

[rsc]

We were away last week. This issue will move to likely accept this week.

We should leave the old mux behavior unchanged. It is still accessible from a GODEBUG but we want people to move off it. We are not updating it, and eventually it may be removed. Adding new API to it gives the opposite impression.

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add a Pattern string field to net/http.Request.
ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux.
In Go 1.21 compatibility mode the field is unset.
The field is otherwise left untouched. Third-party routers may set it if they wish.

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to add a Pattern string field to net/http.Request.
ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux.
In Go 1.21 compatibility mode the field is unset.
The field is otherwise left untouched. Third-party routers may set it if they wish.