Python, Construct: add missing imports of KaitaiStream

[Mingun]

This is one possible way to fix tests. The problem lies in two planes:

• expression translator can generate access to static methods of KaitaiStream, but tests does not import KaitaiStream unconditionally
• Construct also does not import KaitaiStream unconditionally, so generated parsers have missing imports

Example of expr_enum.ksy generated for Construct

from construct import *
from construct.lib import *
import enum
from kaitaistruct import KaitaiStream #<<<<< new import

class expr_enum__animal(enum.IntEnum):
	dog = 4
	cat = 7
	chicken = 12
	boom = 102

expr_enum = Struct(
	'one' / Int8ub,
	'const_dog' / Computed(lambda this: KaitaiStream.resolve_enum(ExprEnum.Animal, 4)),
	'derived_boom' / Computed(lambda this: KaitaiStream.resolve_enum(ExprEnum.Animal, this.one)),
	'derived_dog' / Computed(lambda this: KaitaiStream.resolve_enum(ExprEnum.Animal, this.one - 98)),
)

_schema = expr_enum

The current fix creates a duplicated imports in Python files:

Example of expr_enum.ksy generated for Python

# This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild
# type: ignore

import kaitaistruct
from kaitaistruct import KaitaiStruct, KaitaiStream, BytesIO
from enum import IntEnum
from kaitaistruct import KaitaiStream #<<<<< new import


if getattr(kaitaistruct, 'API_VERSION', (0, 9)) < (0, 11):
    raise Exception("Incompatible Kaitai Struct Python API: 0.11 or later is required, but you have %s" % (kaitaistruct.__version__))

class ExprEnum(KaitaiStruct):

    class Animal(IntEnum):
        dog = 4
        cat = 7
        chicken = 12
        boom = 102
    def __init__(self, _io, _parent=None, _root=None):
        self._io = _io
        self._parent = _parent
        self._root = _root if _root else self
        self._read()

    def _read(self):
        self.one = self._io.read_u1()

    @property
    def const_dog(self):
        if hasattr(self, '_m_const_dog'):
            return self._m_const_dog

        self._m_const_dog = KaitaiStream.resolve_enum(ExprEnum.Animal, 4)
        return getattr(self, '_m_const_dog', None)

    @property
    def derived_boom(self):
        if hasattr(self, '_m_derived_boom'):
            return self._m_derived_boom

        self._m_derived_boom = KaitaiStream.resolve_enum(ExprEnum.Animal, self.one)
        return getattr(self, '_m_derived_boom', None)

    @property
    def derived_dog(self):
        if hasattr(self, '_m_derived_dog'):
            return self._m_derived_dog

        self._m_derived_dog = KaitaiStream.resolve_enum(ExprEnum.Animal, self.one - 98)
        return getattr(self, '_m_derived_dog', None)

The other possible solution is import KaitaiStream unconditionally in tests and Construct, like in the Python:

kaitai_struct_compiler/shared/src/main/scala/io/kaitai/struct/languages/PythonCompiler.scala

Line 52 in 128e179

importList.add(s"from kaitaistruct import $kstructName, $kstreamName, BytesIO")

Fixes tests:

• Python:
  • ExprBytesNonLiteral
• Construct:
  • EnumOfValueInst
  • ExprBytesNonLiteral
  • ExprBytesOps
  • ExprEnum

[generalmimon]

• Construct also does not import KaitaiStream unconditionally, so generated parsers have missing imports

Source files generated for the construct target must never import KaitaiStream (or anything from our Python runtime library), the only thing they can import is Construct. Same thing as in kaitai-io/kaitai_struct_tests#122 (comment).

[Mingun]

the only thing they can import is Construct.

Sorry, didn't read this long API reference.

What do you mean? Construct has its own compiler that allows only Construct imports? Generated files calls methods from KS runtime (why it couldn't be a Construct runtime? If the code the same, why this runtime cannot be the same as Python runtime?)