Allowing httr::write_disk to actually create files
#121
Comments
|
Thanks for the issue! I'll take a look and get back to you soon |
|
It may well be out of scope but it wasn't obvious that it wouldn't work. I'm not trying to record an existing API - I can see that |
|
I assume you saw See also #57 and ropensci/vcr#81 I think the issue is that for this to work with See also the function |
|
If I understand the
Is that right? My use case seems like it might be outside the package design and scope. It would either need an extra argument ( |
I don't think it's outside what library(httr)
f <- tempfile(fileext = ".json")
file.exists(f)
stub_request("get", "https://hb.opencpu.org/get") %>%
to_return(
body = mock_file(path = f, payload = "{\"foo\": \"bar\"}"),
headers = list('content-type' = "application/json")
)
file.exists(f)
out <- GET("https://hb.opencpu.org/get", write_disk(f))
out
file.exists(f)
out$content
readLines(out$content)
content(out, "text", encoding = "UTF-8")See how |
|
That's really useful - I hadn't quite got that mechanism straight in my head. I was trying to avoid having to hold the file contents in memory in the stub registry, but the payloads aren't that big! The thing that is more of an issue is that I didn't mention that some of my payloads are binary files. The |
|
I think i could support binary too. Would adding that be the last thing you need for this to be useable? |
|
Yes - I think that would do it. For the moment, I've got a solution working using a really basic A more general comment is that I found it hard to understand where the content of a response is coming from / going to. If the use case is simply to get the content itself - which is probably 99% of the time - then that doesn't matter, but if did for me. The basic pattern seems to be that the type of the
I'm not quite sure what I'm asking for here but I found some of those outcomes surprising. One approach might be something like a separation in library(httr)
library(webmockr)
enable()
mock_file <- tempfile(fileext = ".json")
out_file <- tempfile(fileext = ".json")
source_file <- tempfile(fileext = ".json")
cat("{\"foo\": \"bar\"}", file = source_file)
# 4 x 2 combinations of possible to_return bodies and httr output settings
bodies <- list(
"source_file" = source_file,
"source_path" = source_file,
"char" = "{\"foo\": \"bar\"}",
"mock_file" = mock_file(path = mock_file, payload = "{\"foo\": \"bar\"}")
)
httr_out <- list(
"mem" = write_memory(),
"disk" = write_disk(out_file)
)
# filename/role lookup
files <- list("mock_file", "out_file", "source_file")
names(files) <- c(mock_file, out_file, source_file)
stub_registry_clear()
# Loop over combinations
for (idxb in seq_along(bodies)) {
for (idxo in seq_along(httr_out)) {
# Get inputs
bod <- bodies[idxb]
out <- httr_out[idxo]
if (names(bod) == "source_file") {
body <- file(bod[[names(bod)]])
} else {
body <- bod[[names(bod)]]
}
# Create stub with requested body
stub <- stub_request("get", "https://hb.opencpu.org/get") %>%
to_return(
body = body,
headers = list("content-type" = "application/json")
)
# GET response with requested output
response <- GET("https://hb.opencpu.org/get", out[[names(out)]])
# Report
content_is_httr_path <- inherits(response$content, "path")
if (content_is_httr_path) {
actual_content <- response$content
} else {
actual_content <- content(response, "text", encoding = "UTF-8")
}
cat(
"body: ", names(bod),
"\nhttr requested output: ", names(out),
"\nroundtrip success: ",
all.equal(content(response), list("foo" = "bar")),
"\ncontent is httr path: ", content_is_httr_path,
"\nactual content: ", actual_content,
"\nwhich path: ", files[[actual_content]],
"\n-------------------\n"
)
remove_request_stub(stub)
}
}
body: source_file
httr requested output: mem
roundtrip success: TRUE
content is httr path: TRUE
actual content: /var/folders/62/cz0ctby96lq7ggnytfbtd__40000gn/T//Rtmp1wVdDX/filef55676e491a7.json
which path: source_file
-------------------
body: source_file
httr requested output: disk
roundtrip success: TRUE
content is httr path: TRUE
actual content: /var/folders/62/cz0ctby96lq7ggnytfbtd__40000gn/T//Rtmp1wVdDX/filef55676e491a7.json
which path: source_file
-------------------
body: source_path
httr requested output: mem
roundtrip success: TRUE
content is httr path: FALSE
actual content: /var/folders/62/cz0ctby96lq7ggnytfbtd__40000gn/T//Rtmp1wVdDX/filef55676e491a7.json
which path: source_file
-------------------
body: source_path
httr requested output: disk
roundtrip success: TRUE
content is httr path: FALSE
actual content: /var/folders/62/cz0ctby96lq7ggnytfbtd__40000gn/T//Rtmp1wVdDX/filef55676e491a7.json
which path: source_file
-------------------
body: char
httr requested output: mem
roundtrip success: TRUE
content is httr path: FALSE
actual content: {"foo": "bar"}
which path:
-------------------
body: char
httr requested output: disk
roundtrip success: TRUE
content is httr path: FALSE
actual content: {"foo": "bar"}
which path:
-------------------
body: mock_file
httr requested output: mem
roundtrip success: TRUE
content is httr path: TRUE
actual content: /var/folders/62/cz0ctby96lq7ggnytfbtd__40000gn/T//Rtmp1wVdDX/filef5566e9582e8.json
which path: mock_file
-------------------
body: mock_file
httr requested output: disk
roundtrip success: TRUE
content is httr path: TRUE
actual content: /var/folders/62/cz0ctby96lq7ggnytfbtd__40000gn/T//Rtmp1wVdDX/filef5566e9582e8.json
which path: mock_file
------------------- |
|
Okay, I can make a change for binary support |
hmm, You're right there is a discrepancy between what body is returned and what you request httr to do wrt disk/memory. The issue is that we need to tell webmockr what to do explicitly - the http client package (httr, crul) does not and should not determine what is returned in the mocked response other than indirectly - we grab some things from the request object to put into the response. In this thread the focus is on mocking the response body when the http client library writes to disk. Unfortunately, the Ruby library webmock this is based off does not handle this use case of an http library writing directly to disk. It just doesn't happen in Ruby. They make you do an http request, then write to disk if you want to do that. So there's no nicely paved route for how to handle this. |
|
Many thanks. Interesting that Ruby forces you to save to disk manually - there's a lot that can go wrong there with
|
Yeah, but I think actually R may be the outlier in this case - httr and friends in R definitely make things simpler for users compared to other langs.
I'll have a think about it |
sorry about the delay on this, working on integrating httr2 here and in vcr, and it's rather complicated |
|
@sckott I have a home rolled solution (with much reduced functionality and robustness) so |
Session Info
I'm trying to use
webmockrto mock an API for my safedata package, to allow code examples and vignette building in packaging to run without requiring a running server providing the API.The issue I'm having is that I'd like to have request stub actually create a file from the response body. That is how
httr::GETwithwrite_diskworks without mocking - the response body is written to the named file. However, when the request is stubbed then I thinkwrite_diskis ignored and replaced with whatever is set in theto_returnbody, which could be an existing file containing the response or some other in memory response. Or at least I think that is how it works (see R code below).I can see why this is used - there's no duplication of content and running
httr:contenton the response object gets you the body - but in my case I want to use the request to build up a local file structure in the same way that it would if I wasn't mocking the API.Is there a way to get
webmockrto provide a response body from a file (or directly!) and allowhttrto write it out to a new file?The text was updated successfully, but these errors were encountered: