CLI tool for generating code

[aaronpowell]

Fixes #192

[atifaziz]

I think the CLI should start out really simple, by taking a single (unnamed argument) Python source/file as input and emitting the generated C# code to standard output. Ideally, also, if no file argument is provided then the Python source should be read from standard input.

The set of problems that I see with the current interface is that it's not very friendly to overall shell and ad-hoc usage. For example:

• There is no exit code suggesting success of failure so it can't be composed with && or || in many shell languages like Bash and cmd.exe.
• Shell globbing where foo/*.py bar/*.py gets expanded by the shell can't be used.
• Usage with xargs wouldn't be obvious as it doesn't just take file names as arguments.
• The generated code cannot be harnessed in a pipeline without additional effort and disk footprint.
• Standard output is used to emit diagnostic or error messages.

All of the above could be fixed with some simple changes that are easier to describe, visualise and apply as a diff:

diff --git a/src/CSnakes.CLI/Program.cs b/src/CSnakes.CLI/Program.cs
index 07fcac7..48dad50 100644
--- a/src/CSnakes.CLI/Program.cs
+++ b/src/CSnakes.CLI/Program.cs
@@ -1,67 +1,52 @@
 using CSnakes;
 using Microsoft.CodeAnalysis.Text;
+using System.Collections.Immutable;
 using System.CommandLine;
+using System.Runtime.CompilerServices;
 
-var fileOption = new CliOption<FileInfo?>("--file", "-f")
+var inputsArg = new CliArgument<FileSystemInfo[]>("FILE")
 {
-    Description = "Path to the Python file to generate the C# code from."
-};
-
-var directoryOption = new CliOption<DirectoryInfo?>("--directory", "-d")
-{
-    Description = "Path to the directory containing the Python files to generate the C# code from."
+    Arity = ArgumentArity.ZeroOrMore
 };
 
 var outputOption = new CliOption<DirectoryInfo?>("--output", "-o")
 {
     Description = "Path to the directory to output the C# files to.",
-    Required = true
 };
 
-var root = new CliRootCommand("csnakes -f <file>")
+var root = new CliRootCommand("Creates C# wrapper to call Python code.")
 {
-    fileOption,
+    inputsArg,
     outputOption,
-    directoryOption
 };
 
 root.SetAction(result =>
 {
-    FileInfo? fileInfo = result.GetValue(fileOption);
     DirectoryInfo? outputInfo = result.GetValue(outputOption);
-    DirectoryInfo? directoryInfo = result.GetValue(directoryOption);
 
-    if (fileInfo is null && directoryInfo is null)
-    {
-        Console.WriteLine("You must provide either a file or a directory.");
-        return;
-    }
+    var inputs = result.GetValue(inputsArg) ?? [];
 
-    if (fileInfo is not null && directoryInfo is not null)
-    {
-        Console.WriteLine("You must provide either a file or a directory, not both.");
-        return;
-    }
+    ImmutableArray<FileInfo> files = [..
+        from fsi in inputs
+        from file in fsi switch
+        {
+            FileInfo file => [file],
+            DirectoryInfo directory => directory.GetFiles("*.py"),
+            var info => throw new SwitchExpressionException(info)
+        }
+        select file];
 
-    if (outputInfo is null)
+    if (outputInfo is not null)
     {
-        Console.WriteLine("You must provide an output file.");
-        return;
+        outputInfo.Create();
     }
-
-    FileInfo[] files = (fileInfo, directoryInfo) switch
+    else if (files.Length > 1)
     {
-        (not null, _) => [fileInfo],
-        (_, not null) => directoryInfo.GetFiles("*.py"),
-        _ => throw new InvalidOperationException()
-    };
-
-    if (outputInfo.Exists)
-    {
-        outputInfo.Delete(true);
+        Console.Error.WriteLine("You must provide an output directory when processing multiple files.");
+        return 1;
     }
 
-    outputInfo.Create();
+    var success = true;
 
     foreach (var file in files)
     {
@@ -69,16 +54,30 @@ root.SetAction(result =>
         var pascalFileName = PythonStaticGenerator.GetPascalFileName(Path.GetFileNameWithoutExtension(file.Name));
 
         if (PythonStaticGenerator.TryGenerateCode((error) =>
+            {
+                Console.Error.WriteLine($"{file.FullName}:{error.StartLine}:{error.StartColumn}: error: {error.Message}");
+            }, pascalFileName, Path.GetFileNameWithoutExtension(file.Name), sourceFile, out var source))
         {
-            Console.WriteLine($"Error: {error.Message}");
-        }, pascalFileName, Path.GetFileNameWithoutExtension(file.Name), sourceFile, out var source))
+            if (outputInfo is { FullName: var outputDirPath })
+            {
+                string outputFilePath = Path.Combine(outputDirPath, $"{pascalFileName}.py.cs");
+                Console.Error.WriteLine($"{file.FullName} > {outputFilePath}");
+                File.WriteAllText(outputFilePath, source);
+            }
+            else
+            {
+                Console.Write(source); // assume source is new-line terminated
+                Console.Out.Flush();
+            }
+        }
+        else
         {
-            string outputFileName = $"{pascalFileName}.py.cs";
-            File.WriteAllText(Path.Combine(outputInfo.FullName, outputFileName), source);
+            success = false;
         }
     }
 
-    Console.WriteLine($"Generated code for {files.Length} files.");
+    Console.Error.WriteLine($"Generated code for {files.Length} files.");
+    return success ? 0 : 1;
 });
 
 await new CliConfiguration(root).InvokeAsync(args);

---

PS If it helps, I'd be happy to provide these changes as a PR on top of this one.

[atifaziz]

nit: Use --version-suffix that's natively supported by dotnet pack (for consistency):

Suggested change

	run: dotnet pack --no-build -c ${{ env.DOTNET_CONFIGURATION }} -p:VersionSuffix='alpha.${{ github.run_number }}'
	run: dotnet pack --no-build -c ${{ env.DOTNET_CONFIGURATION }} --version-suffix 'alpha.${{ github.run_number }}'

[atifaziz]

nit: Use --version-suffix that's natively supported by dotnet pack (for consistency):

Suggested change

	run: dotnet pack --no-build -c ${{ env.DOTNET_CONFIGURATION }} -p:VersionSuffix='beta.${{ github.run_number }}'
	run: dotnet pack --no-build -c ${{ env.DOTNET_CONFIGURATION }} --version-suffix 'beta.${{ github.run_number }}'

[aaronpowell]

I think the CLI should start out really simple, by taking a single (unnamed argument) Python source/file as input and emitting the generated C# code to standard output. Ideally, also, if no file argument is provided then the Python source should be read from standard input.

I disagree with this from the ergonomics of a .NET developer workflow, after all, the target of this is .NET developers who are wanting to leverage some Python code in their app.

So, when I look at it through that view of things, we see CLI tools taking named arguments rather than streaming.

[atifaziz]

I disagree with this from the ergonomics of a .NET developer workflow, after all, the target of this is .NET developers who are wanting to leverage some Python code in their app.

I was coming at this from the objective angle of a CLI tool that's simply a code generator without thinking about a specific audience. That said, I think that .NET developers often work with many different tools, and they likely have a diverse set of workflows. Many .NET projects have build pipelines, automation scripts, and integration with other languages or tools (like Python in this case), so chances are high that .NET developers are accustomed to various flavours of CLI tools and shells. Many commands and sub-commands of the .NET CLI also take plain (as opposed to named) arguments where it's super obvious, so the suggestions here were in a similar spirit to:

dotnet build [<PROJECT | SOLUTION>...] [options]
dotnet run [<applicationArguments>...] [options]
dotnet pack [<PROJECT | SOLUTION>...] [options]
dotnet new [<template-short-name> [<template-args>...]] [options]
dotnet add [<PROJECT>] package <PACKAGE_NAME> [options]

As for whether anyone will ever combine with other tools, I think working with Git-tracked files is often important, so being able to splat like this in PowerShell (assuming that's what most .NET developers use):

csnakes @(git ls-files src/Integration.Tests/*.py)

or piping like this (although not a strong case as this would work fine with -f too):

git ls-files src/Integration.Tests/*.py | % { csnakes $_ }

is really super helpful and productive. A .NET developer on WSL might use xargs if they're familiar with it:

git ls-files src/Integration.Tests/*.py | xargs -t -n 1 csnakes

So, when I look at it through that view of things, we see CLI tools taking named arguments rather than streaming.

I find that the source generator already covers the .NET developer audience well with tight integration into the compiler pipeline and IDE. The CLI tool should address and enable different scenarios. My goal here was to give some feedback to consider, but ultimately it's your call.

[aaronpowell]

I've just had a go at implementing this and I find the ergonomics of working with a CLI like this is really clunky, and I also wonder if it's not inefficient as each time we pass in a file we have to start up the CLI again because it's only handling one file at a time.

With the example of piping the output of a globbing query to the CLI, it's not much more to change

git ls-files src/Integration.Tests/*.py | % { csnakes $_ }

to

git ls-files src/Integration.Tests/*.py | % { csnakes -f $_ }

and it'd be similar with the xargs approach.

I can see the value in having -o as an optional flag, I'll make that change, and if it's not present it'll redirect to stdout.

Maybe for the next revision, adding globbing support to the -f or -d flags would make sense, as then we could the value in that, but I think it's more of an advanced feature than a common use-case.

@tonybaloney - given the CLI request was from you, did you have specific scenarios in mind of how someone would use it?

[atifaziz]

I also wonder if it's not inefficient as each time we pass in a file we have to start up the CLI again because it's only handling one file at a time.

That's true and which is why you'd just splat all the files on the command line, like this:

csnakes @(git ls-files src/Integration.Tests/*.py)

You can control splatting of one or more arguments with xargs too using its -n/--max-args argument. -n 1 will cause one file to be processed at a time whereas the default is to put as many as possible.

[aaronpowell]

Ok, figured out how to make globbing/splats work in System.CommandLine without much overhead (the docs are not helpful), and it's working on my tests.