Windows Ruby 3.1 - Open3 #capture2 causes IRB console to become unresponsive

[johnnyshields]

I have also filed a ticket with ruby/open3 repo here: ruby/open3#9, but this seems more like a Windows-specific issue as it works on Linux/Mac.

The following script will cause IRB console to become unresponsive on Ruby 3.1 on Windows installed by the Windows Ruby Installer x64-mingw-ucrt. This is using the version of Open3 packaged with Ruby (0.1.1)

$> irb

require 'open3'
Open3.capture2('ver')

# after this, the IRB console becomes unresponsive

[mohits]

Hi @johnnyshields - I did a quick check with your sample on my side, and it seems to work as expected. This is in the Windows console (cmd).

$ ruby -v
ruby 3.1.1p18 (2022-02-18 revision 53f5fc4236) [x64-mingw-ucrt]

$ irb
irb(main):001:0> require 'open3'
=> true
irb(main):002:0> Open3.capture2('ver')
=> ["\nMicrosoft Windows [Version 10.0.19044.1766]\n", #<Process::Status: pid 13272 exit 0>]
irb(main):003:0> (0..2).each {|x| puts "Hello #{x}"}
Hello 0
Hello 1
Hello 2
=> 0..2
irb(main):004:0> exit

Just adding this here in case it helps to debug what the difference might be.

[johnnyshields]

@mohits thanks for checking this, this is very useful. Hmmm very strange. Possibly something specific to my system.

Quite bizarre tho as Ruby 3.0 and earlier don't have the same issue.

[mohits]

@johnnyshields - I checked both Ruby 3.0 and Ruby 3.1 on my system (though I only posted the result from 3.1) and both worked fine. The installation is also bog-standard - https://notepad.onghu.com/2022/ruby3.1-windows-day0-install-hello_world/ - nothing special in it.

[johnnyshields]

@mohits I appreciate it.

[MSP-Greg]

The following script will cause IRB console to become unresponsive on Ruby 3.1

Testing with Ruby 3.1.2 and Ruby master (ruby 3.2.0dev (2022-06-26T02:16:11Z master a782d76fbe) [x64-mingw-ucrt]), I have the same result.

[P3t3rU5]

same is happening to me and it is making nokigiri installation fail

> gem install nokogiri
Temporarily enhancing PATH for MSYS/MINGW...
Building native extensions. This could take a while...
#<Thread:0x0000013ef5ede448 C:/Programming/Ruby/rubyinstaller-3.2.0-1-x64/lib/ruby/3.2.0/open3.rb:404 run> terminated with exception (report_on_exception is true):
C:/Programming/Ruby/rubyinstaller-3.2.0-1-x64/lib/ruby/3.2.0/open3.rb:404:in `read': stream closed in another thread (IOError)
        from C:/Programming/Ruby/rubyinstaller-3.2.0-1-x64/lib/ruby/3.2.0/open3.rb:404:in `block (2 levels) in capture2e'
ERROR:  Interrupted

[ccmywish]

Test report: gem install nokogiri on Windows 11 (10.0.22621.1105) with Ruby 3.2.0-1 works fine.

I think this may relate to some specific Windows versions.

[P3t3rU5]

Seems like MSYS was the issue here - installed on top of my previous installation and it started working

[P3t3rU5]

nevermind... seemed to work but when doing bundler update it gets stuck trying to install nokogiri

[johnnyshields]

This issue is still persisting with the officially released Windows RubyInstaller for Ruby 3.2.0

[OursCodeur]

Hey, I can confirm I have the issue as well here - Windows 11 [Version 10.0.22621.1702] and ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt] .

[MSP-Greg]

Not sure if this is an Open3 issue, as the following work. Might be an irb issue?

ruby -e "require 'open3'; puts Open3.capture2('ver')"
ruby -e "require 'open3'; Thread.new { puts Open3.capture2('ver') }.join"

[OursCodeur]

What started the investigation on my side was using mjml-rails in a regular project on Windows - which relies on Open3 to run MJML itself (here). Capture3 causes ruby to hang up indefinitely, which I can reproduce in irb.

I can also confirm @MSP-Greg 's commands do not cause the issue.

Hope the additional context helps somehow.

[johnnyshields]

Technically speaking IRB does not “crash” either, but its state is altered so that it no longer accepts inputs. So the fact that the ruby -e commands don’t crash maybe doesn’t tell us a whole lot.

[OursCodeur]

Trying to work my way around Open3, I've also been able to reproduce the IRB hang with systemu('env') .

[johnnyshields]

FYI this issue still happens on Windows Ruby Installer for Ruby 3.3.1

[johnnyshields]

@MSP-Greg @larskanis FYI this issue remains on my "Windows Ruby Unsolved Mysteries" list. It would be great if we could try to figure it out, at the moment I have to edit gems like google-apis-core to remove usages of Open3.capture2 each time I install them.

[adfoster-r7]

We're hitting this issue too on two different Windows 11 versions Ruby versions:

C:\Users\a>systeminfo
...
OS Name:                   Microsoft Windows 11 Home
OS Version:                10.0.22000 N/A Build 22000

C:\Users\a>ruby --version
ruby 3.3.3 (2024-06-12 revision f1c7b6f435) [x64-mingw-ucrt]

And:

C:\Users\a>systeminfo
...
OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22631 N/A Build 22631

PS C:\Users\a\Desktop> ruby --version
ruby 3.1.5p252 (2024-04-23 revision 1945f8dc0e) [x64-mingw-ucrt]

Is there anything I could do to help with debugging this issue? Happy to try and help 👀

[adfoster-r7]

For context we're hitting the issue outside of irb with reline:

require 'open3'
require 'reline'

Open3.capture3('git status')
line = Reline.readline('hello > ')

Outcome:

c:\example>ruby foo.rb
▽hello > h

And the terminal hangs; however removing the Open3 call works as expected:

c:\example>ruby foo.rb
hello >

---

Poked a bit further, and it looks like this replicates with some of the lower level building blocks of popen:

require 'open3'
require 'reline'

def popen_bug(cmd)
    opts = {}

    in_r, in_w = IO.pipe
    opts[:in] = in_r
    in_w.sync = true
  
    out_r, out_w = IO.pipe
    opts[[:out, :err]] = out_w
  
    pid = spawn(*cmd, opts)
    wait_thr = Process.detach(pid)
    
    wait_thr.value
end

popen_bug("dir")
line = Reline.readline('hello > ')

Output broken and hanging:

c:\foo>ruby foo.rb
▽hello > h

But after commenting out the in pipe config option, the issue disappears for me:

require 'open3'
require 'reline'

def popen_bug(cmd)
    opts = {}

    # Commenting this out causes things to work again:
    # in_r, in_w = IO.pipe
    # opts[:in] = in_r
    # in_w.sync = true
  
    out_r, out_w = IO.pipe
    opts[[:out, :err]] = out_w
  
    pid = spawn(*cmd, opts)
    wait_thr = Process.detach(pid)
    
    wait_thr.value
end

popen_bug("dir")
line = Reline.readline('hello > ')

[mohits]

hi @adfoster-r7 - this is strange.

On Windows 10, using cmd, I was able to run the script that you had a problem with (including in_r) and on both Ruby 3.2-x64 and Ruby 3.3-x64, I did not have any issue.

I got hello > and was able to type in the "this is all ok" text and press enter so the script completed.

$ ruby popen.rb
hello > this is all ok

I don't have Windows 11, so I can't check if it's related to that.

Ruby 3.2 is using open3 (default: 0.1.2)
Ruby 3.3 is using open3 (default: 0.2.1)