Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

context.createRequest didn't set "GetBody" properly. #24

Open
elonzh opened this issue Mar 17, 2017 · 2 comments
Open

context.createRequest didn't set "GetBody" properly. #24

elonzh opened this issue Mar 17, 2017 · 2 comments
Labels

Comments

@elonzh
Copy link

elonzh commented Mar 17, 2017

The context package wrap the http.Request.Body with a internal type utils.nopCloser in createRequest and replace body with a new type context.contextReadCloser when we try to update values in context.

Unfortunately, the GetBody method for http.Request didn't been set.

The std lib net/http set GetBody for us when we use http.NewRequest and this solution covered most cases. L788-L830 in Go1.8

	if body != nil {
		switch v := body.(type) {
		case *bytes.Buffer:
			req.ContentLength = int64(v.Len())
			buf := v.Bytes()
			req.GetBody = func() (io.ReadCloser, error) {
				r := bytes.NewReader(buf)
				return ioutil.NopCloser(r), nil
			}
		case *bytes.Reader:
			req.ContentLength = int64(v.Len())
			snapshot := *v
			req.GetBody = func() (io.ReadCloser, error) {
				r := snapshot
				return ioutil.NopCloser(&r), nil
			}
		case *strings.Reader:
			req.ContentLength = int64(v.Len())
			snapshot := *v
			req.GetBody = func() (io.ReadCloser, error) {
				r := snapshot
				return ioutil.NopCloser(&r), nil
			}
		default:
			// This is where we'd set it to -1 (at least
			// if body != NoBody) to mean unknown, but
			// that broke people during the Go 1.8 testing
			// period. People depend on it being 0 I
			// guess. Maybe retry later. See Issue 18117.
		}
		// For client requests, Request.ContentLength of 0
		// means either actually 0, or unknown. The only way
		// to explicitly say that the ContentLength is zero is
		// to set the Body to nil. But turns out too much code
		// depends on NewRequest returning a non-nil Body,
		// so we use a well-known ReadCloser variable instead
		// and have the http package also treat that sentinel
		// variable to mean explicitly zero.
		if req.GetBody != nil && req.ContentLength == 0 {
			req.Body = NoBody
			req.GetBody = func() (io.ReadCloser, error) { return NoBody, nil }
		}
	}

That's mean, even we sent our request, we still can get the request body if we want.

For gentleman, the request was built when dispatching, we can't get the request body before we sent it compared with http.NewRequest(although this behavior is useless ).

More importantly, we also can't get the request body !

That's mean, it's tricky if some jobs base on the request we sent, like storing the response by request fingerprint.

Here is a example to reproduce the problem.

package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"net/http"

	"gopkg.in/h2non/gentleman.v1"
)

func main() {
	body := bytes.NewBufferString("Body Hell!")
	request, _ := http.NewRequest("POST", "http://httpbin.org/post", body)
	res, _ := http.DefaultClient.Do(request)
	// get request body after sent
	if body, _ := ioutil.ReadAll(res.Request.Body); body != nil {
		fmt.Println("the origin Body is empty:", string(body[:]))
	}
	bodyClone, _ := res.Request.GetBody()
	if body, _ := ioutil.ReadAll(bodyClone); body != nil {
		fmt.Println("Body still can be gotten by GetBody:", string(body[:]))
	}
	req := gentleman.NewRequest()
	req.Method("POST")
	req.URL("http://httpbin.org/post")
	req.BodyString("Body Hell!")
	fmt.Println("the req.Context.Request.GetBody is nil because it didn't be set in `createRequest`:", req.Context.Request.GetBody == nil)
	gres, _ := req.Do()
	if body, _ := ioutil.ReadAll(gres.Context.Request.Body); body != nil {
		fmt.Println("the Body is empty and we can'y get cached body by GetBody:", string(body[:]))
	}

Output:

the origin Body is empty: 
Body still can be gotten by GetBody: Body Hell!
the req.Context.Request.GetBody is nil because it didn't be set in `createRequest`: true
the Body is empty and we can'y get a cache for GetBody: 
@elonzh
Copy link
Author

elonzh commented Mar 17, 2017

This issue is similar to #16

@h2non h2non added the question label Mar 17, 2017
@h2non
Copy link
Owner

h2non commented Mar 17, 2017

Alright, let's separate concerns here.

Regarding GetBody(), yes, this is indeed an issue and should be fixed. Let me know if you're interested in providing a PR.

Regarding reading the request body, and considering your example code, you're not using gentleman idiomatically. Let me explain details:

gentleman is a plugin-oriented, middleware-driven toolkit where everything is a composable entity that can be plugged into any outgoing HTTP traffic flow in both client or single HTTP request level. The reason of this is because in gentleman everything is defined in a lazy way based on some sort of observables hooks, so where when you call, for instance, BodyString() method, you're not technically binding the body yet into the HTTP request object and that's why you can't introspect it.

In order to achieve that, you should use the gentleman approach and attach a custom middleware function or plugin that can observe the HTTP traffic. You can achieve that by observing both outgoing and incoming HTTP traffic flows. See supported middleware/plugin phases here.
You can also see some plugins for usage reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants