repro.common.io#
- repro.common.io.read_jsonl_file(file_path, single_line=False)#
Loads a jsonl file and returns a list corresponding to each line in the file.
- Parameters
file_path (str) – The file to load the data from
single_line (bool, default=False) – Indicates all of the json objects are on a single line (i.e., not separated by a newline character, but separated by “”)
- Returns
The items, where each item corresponds to one line
- Return type
List
- repro.common.io.write_to_jsonl_file(items, key, file_path, flatten=False, separator=None)#
Writes the items in items to a jsonl file at file_path. Each item will correspond to one line. The item will be the value for the key in each json. If flatten is True, each item will first be flattened using separator.
- Parameters
items (List[Union[str, List[str]]]) – The items to serialize
key (str) – The key for the items
file_path (str) – The path to the file where the output should be written
flatten (bool, default=False) – Indicates whether the items should first be flattened
separator (str, default=None) – The separator for flattening the items
- repro.common.io.write_to_text_file(items, file_path, separator=None)#
Writes the items in items to a text file with one item per line. If an individual item is a list, it is first flattened with the separator.
- Parameters
items (List[Union[str, List[str]]]) – The items to write to a file.
file_path (str) – The path to the file where the output should be written
separator (str, default=None) – The separator to use to join the items