Is there a way to know whether data has been read in source connector

Connector Development
,
1

Hi …Is there a way to know whether data has been read in the source connector.
Eg:

Following is the read method of source-file connector:
def read(
self, logger: AirbyteLogger, config: Mapping, catalog: ConfiguredAirbyteCatalog, state_path: Mapping[str, any]
) → Generator[AirbyteMessage, None, None]:
“”“Returns a generator of the AirbyteMessages generated by reading the source with the given configuration, catalog, and state.”“”
client = self._get_client(config)
fields = self.selected_fields(catalog)
name = client.stream_name

    logger.info(f"Reading {name} ({client.reader.full_url})...")
    try:
        for row in client.read(fields=fields):
            record = AirbyteRecordMessage(stream=name, data=row, emitted_at=int(datetime.now().timestamp()) * 1000)
            yield AirbyteMessage(type=Type.RECORD, record=record)
    except Exception as err:
        reason = f"Failed to read data of {name} at {client.reader.full_url}: {repr(err)}\n{traceback.format_exc()}"
        logger.error(reason)
        raise err

Do we have any handler here which confirms that the data has been read successfully from the file?

Based on that I would like to have post read logic.
Please suggest.

2

Can you explain your use case? Are you only trying to print the number of records read?
Airbyte doesn’t print records for security reason.