Partitioning Python code that generates different types of things

Your second approach fails when foo, bar and baz are not of three different types, so it is actually a bad idea. Heck, in a dynamically typed language like Python, it is not even guaranted that each foo object will have the same type.

What you can do without changing the initial scan method: you can implement a class FooCollector derived from ScanHandler, where onFoo just collects all passed foo objects in a list, and onBar as well as onBaz stay empty. Then call scan(someInput, fooCollector) and ask fooCollector afterwards for the list of foo objects it got. This is dead simple and does not require any type checks.

You could then implement scanner.getAllFoos this way:

class Scanner:
    ...
    def getAllFoos(someInput):
         fooHandler = FooHandler()
         scan(someInput,fooHandler)
         return fooHandler.listOfCollectedFoos

As a variant of this, you may implement a FooHandler in a similar manner, which does not just fill a list, but does the foo processing directly in-place.

Where I would also take a deeper look into: the section

... complicated logic ... 

should be refactored to a function which returns a result which makes he following conditionals if ... something ...: simple. This gives you the option to provide an extra function scanForFoos sharing the same logic as scan without duplicating it. Something along the lines of

def scan(someInput, handler : ScanHandler):
    for chunk in someInput:
        result=callToComplicatedLogic(...)
        if result.conditionForFoo():
            foo = result.getFoo()
            handler.onFoo(foo)
        if result.conditionForBar():
            bar = result.getBar()
            handler.onBar(bar)
        if result.conditionForBaz():
            baz = = result.getBaz()
            handler.onBaz(baz)

def scanForFoos(someInput):
    for chunk in someInput:
        result = callToComplicatedLogic(...)
        if result.conditionForFoo():
            foo = result.getFoo()
            yield foo

This keeps the complicated logic in one place, hence the code DRY, with a little bit more boilerplate, of course. This approach has the advantage that it allows the caller to iterate over a subset of foo object and stop the iteration, lets say, after the first 10 objects, without making the scanner going through the full input data.

Based on your comment on Doc Brown's answer, I think you are looking for something like this:

class Scanner:
    ...
    def scan(someInput, handler : ScanHandler, emit = ('foo', 'bar', 'baz')):
        for chunk in someInput:
            ... complicated logic ...
            if ... something ... and 'foo' in emit:
                foo = ...
                handler.onFoo(foo)
            if ... something else ... and 'bar' in emit:
                bar = ...
                handler.onBar(bar)
            if ... something else ... and 'baz' in emit:
                baz = ...
                handler.onBaz(baz)

If you exclude an emit parameter, it will work as it does now. If you want to just get 'foo' output, you can call like this:

scan(input, scanner, ['foo'])

There are many levels of clever fanciness that you can add to this basic idea. Another simple one would be to do this instead:

def scan(someInput, handler : ScanHandler, emit = lambda x: True):
   for chunk in someInput:
       ... complicated logic ...
       if ... something ... and emit('foo'):
          foo = ...
          handler.onFoo(foo)

Then the foo-only call becomes:

scan(input, scanner, lambda x: x == 'foo')

The advantage here being that your function allows for arbitrary logic. You can add more parameters to add more capabilities.

You might also want to consider adding the emit list (or function) to the Scanner object. This might be useful if the decision of what to emit does not change during the scanners lifetime. There's also nothing preventing you from doing both. That is, setting it on the object and allowing it on the method call. You just need to decide what you want to do if both are set.

Filter

Nothing is faster, than doing nothing.

Andrei Alexandrescu

In general, constructing an object just to discard it is pretty wasteful work. If the construction is fast, and free of side-effects, then you may just go ahead and do it... but do consider filtering.

Filtering is relatively easy, provided you have a fixed list of item kinds/tags you care about:

1. Define an enum, where each element is but an integer.
2. Pass a set of integers -- possibly a bit-set -- to the scan method.

Iteration

The idiomatic way to create an external iterator in Python is to use yield, as you mentioned.

And it can easily be augmented with filtering capabilities:

class Scanner:
    def scan(someInput, filter):
        for chunk in someInput:
            ... complicated logic ...

            if ... something ...:
                if not Tag.FOO in filter:
                    continue

                foo = ...
                yield foo

            if ... something else ...:
                if not Tag.BAR in filter:
                    continue

                bar = ...
                yield bar

            if ... something else ...:
                if not Tag.BAZ in filter:
                    continue

                baz = ...
                yield baz

However, do note that you are losing some information on the client side. When calling a handler method, the method name that is called (onFoo, onBar) is telling the client what it is dealing with, whereas just yielding the item may require the client to "reconstruct" it.

Since Python is dynamic, divining what the kind/tag of the object is may be complicated, and brittle. For a reliable solution, you can:

1. Create a base class with a tag method, which returns the Tag, allowing the client to switch on it.
2. Use a Sum Type to wrap the item.

The former works in any version of Python.

Note: you can also use generator.send(filter) and filter = yield x to allow the user to switch the filter every time they get an item.

Right now, your scanner is mixing three concerns:

1. Iterating over your chunks
2. Parsing the input based on several types
3. Yielding / dispatching input based on several types.

The scanner can be made in an open-closed way, where these concerns are separated:
(note that I am mostly comfortable with C#, but will attempt to produce python)

class Parser():
    @abstractmethod
    def try_parse(self, chunk):
        """Return an object or None if this parser doesn't apply."""
        pass

class FooParser(Parser):
    def try_parse(self, chunk):
        if some_complicated_check:
            return Foo(...)
        return None

class BarParser(Parser):
    def try_parse(self, chunk):
        if other_logic:
            do_processing(chunk)
            return Bar(...)
        return None

Your scanner will become:

class Scanner:
    def __init__(self, parsers):
        self.parsers = parsers

    def scan(self, some_input):
        for chunk in some_input:
            for parser in self.parsers:
                item = parser.try_parse(chunk)
                if item is not None:
                    yield item

You can use the scanner with any parse strategy as you like:

scanner = Scanner([FooParser(), BarParser(), BazParser()])

for item in scanner.scan(some_input):
    ...

or if you care about foo only:

scanner = Scanner([FooParser()])

for foo in scanner.scan(some_input):
    ...

The open-closed principle shows itself when you want to add a new type to be parsed. In that case, you supply the scanner with a new (e.g. QuixParser()), and the scanner code does not need to be modified.

Here's how I would approach this. First define each of those entities as separate classes:

class Foo: ...
class Bar: ...
class Baz: ...

Next define a generic abstract handler per object. Not a single abstract handler for all objects. Like this:

T = TypeVar('T', Foo, Bar, Baz)

class Handler(ABC, Generic[T]):
    @abc.abstractmethod
    def handle(self, obj: T):
        pass

Now we want to store handlers per object type. We will utilize a dict of lists for that. But first lets define a scanner:

class Scanner:
    def __init__(self, handlers):
        # handlers is a dict of lists of Handler instances
        self.handlers = handlers

    def _apply_handlers(self, obj):
        # additional checks and logs here
        obj_type = type(obj)
        for handler in self.handlers[obj_type]:
            handler.handle(obj)

    def run(self, input):
        for chunk in input:
            ... complicated logic ...

            obj = None

            if ... something ...:
                obj = Foo(...)
            if ... something else ...:
                obj = Bar(...)
            if ... something else ...:
                obj = Baz(...)

            if obj is not None:
                self._apply_handlers(obj)
            else:
                # log it?

and finally the corresponding builder:

class ScannerBuilder:
    def __init__(self): 
        self.handlers = {}

    def register(self, obj_type: Type, handler):
        if obj_type not in self.handlers:
            self.handlers[obj_type] = []
        self.handlers[obj_type].push(handler)

    def build(self) -> Scanner:
        return Scanner(self.handlers)

And finally the usage:

builder = ScannerBuilder()
builder.register(Foo, foo_handler)
builder.register(Bar, bar_handler)
builder.register(Baz, baz_handler)
builder.register(Baz, other_baz_handler)
scanner = builder.build()
scanner.run(input)

Looks like clean, separated, easily testable solution to me.

I'd say the simplest (most pythonic?) way to achieve this is to yield tuples consisting of a type marker and the thing itself. You don't need an OOP visitor pattern with ScanHandler instances, not everything needs to be a class.

def scan(someInput):
    for chunk in someInput:
        ... complicated logic ...
        if ... something ...:
            foo = ...
            yield 'Foo', foo
        elif ... something else ...:
            bar = ...
            yield 'Bar', bar
        elif ... something else ...:
            baz = ...
            yield 'Baz', baz

You would typically consume the output with a match statement:

for token in scan(myInput):
    match token:
        case 'Foo', foo:
            """handle the foo"""
        case 'Bar', bar:
            """handle the bar"""
        case other, _
            raise NotImplementedError(f'Unexpected "{other}" token')

But with this basic scan, you can implement all the other patterns on top of this:

def handleScan(someInput, handler: ScanHandler):
    for typ, val in scan(someInput):
        getAttr(handler, 'on'+typ)(val);

If you want to get only foos, you'd write

foos = (val for typ, val in scan(myInput) if typ == 'Foo')

For one-off use, the if statement inside the for loop is totally fine. If you need to scan for only foos in multiple places, and for only bars in multiple other places, add some helper functions that wrap scan. I would however definitely recommend to write separate helper functions (handleScan, scanFoos, scanBars etc), do not try to combine them into one function that will return different types of values depending on how it is called.