added netloc support for hdfs URIs #168
Conversation
…DFS URIs which was against the URI specification
|
@vvaten Thank you for your pull request. It looks good to me. @menshikh-iv Might be a good idea to merge this after our HDFS integration tests are up. What do you think? |
| @@ -355,6 +356,9 @@ class ParseUri(object): | |||
| * file:///home/user/file | |||
| * file:///home/user/file.bz2 | |||
|
|
|||
| NOTE: hdfs://path/file does no longer work as it is against the URI | |||
There was a problem hiding this comment.
Can you add more information to comment (when this happens, what HDFS version affected, etc)?
There was a problem hiding this comment.
This is a generic change that affects all HDFS versions. They do support hdfs://host/path/file URI format. Using hdfs://path/file in smart_open violates this and also violates the URI specification (RFC3986) where the hostname part is always after the '://'. The correct way to refer to local content is hdfs:///path/file instead of hdfs://path/file.
|
Hello is there any way this can be merged ? As part of my work I was trying to read and write to hdfs and traced the bug to this |
removed the special handling of HDFS URIs which was against the URI specification