Starting a process with non-ascii characters in Python 2

I've been recently hitting a Python 2 bug in Windows. When using subprocess, if the process name or any of the arguments contain non-ascii characters, an error like the following is raised:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xc5' in position 5: ordinal not in range(128).

It's the issue on the official bug tracker here, fixed in Python 3 but not Python 2. Looks like the problem stems from using CreateProcessA instead of CreateProcessW. Most of the workarounds listed on this page don't actually work -- specifying a code page will still fail for many unicode characters.

My workaround will work, although it's not very elegant.

def runWithoutWaitUnicode(listArgs):
    # in Windows, non-ascii characters cause subprocess.Popen to fail.
    # https://bugs.python.org/issue1759845
    import subprocess
    if not sys.platform.startswith('win'):
        p = subprocess.Popen(listArgs, shell=False)
        return p.pid
    else:
        import winprocess
        import types
        if isinstance(listArgs, types.StringTypes):
            combinedArgs = listArgs
        else:
            combinedArgs = subprocess.list2cmdline(listArgs)
            
        combinedArgs = unicode(combinedArgs)
        executable = None
        close_fds = False
        creationflags = 0
        env = None
        cwd = None
        startupinfo = winprocess.STARTUPINFO()
        handle, ht, pid, tid = winprocess.CreateProcess(
            executable,
            combinedArgs,
            None, None,
            int(not close_fds),
            creationflags,
            env,
            cwd,
            startupinfo)
        ht.Close()
        handle.Close()
        return pid

It requires only winprocess.py (MIT License), which doesn't have further dependencies (just comment out the unneeded references to QueryInformationJobObject).

Here's a way to simulate running with ShellExecute (like passing shell=True to subprocess): you can pass to cmd.exe. For example, in Windows, to open a file with its default program, you can use runWithoutWaitUnicode([u'cmd', u'/c', u'start', filePath]). This can also be used to open a directory in explorer.

I've placed the full code, including tests, here (search for runWithoutWaitUnicode).