Rdw header missing for variable size ebcdic file

[sobhaw]

I do not have rdw headers ,when it is single copybook it works with variable size true .but with multiple copybooks in used copybooks as well as combined all 4 into single copybook using redefines. Both r giving arrayindexouf of bounds exception as it might not be getting record length correctly for ebcdic file. It is working fine for txt file. I have each record starting with 01ABC1 , 01ABC2, 01ABC3, 01ABC4 which is the record id for 1st , 2nd and so on . How do I tackle this ?

[yruslan]

Hi @sobhaw , please share options you pass to 'spark-cobol', and the copybook, if possible. Array index out of bounds error should not be happening in any circumstance.

[sobhaw]

I used the options like variable_size_occurs schema retention keep original and removed the ascii option and tried to read a binary file the copy had last part as array so once it can't determine the length after reading the first record it gives this issue

[sobhaw]

Not closed i closed it

[sobhaw]

I do not have rdw headers ,when it is single copybook it works with variable size true .but with multiple copybooks in used copybooks as well as combined all 4 into single copybook using redefines. Both r giving arrayindexouf of bounds exception as it might not be getting record length correctly for ebcdic file. It is working fine for txt file. I have each record starting with 01ABC1 , 01ABC2, 01ABC3, 01ABC4 which is the record id for 1st , 2nd and so on . How do I tackle this ?

[sobhaw]

I used the options like variable_size_occurs schema retention keep original and removed the ascii option and tried to read a binary file the copy had last part as array so once it can't determine the length after reading the first record it gives this issue

[yruslan]

There are multiple ways to read variable record length file depending on the way to determine the length of each record. From your description looks like record lengths can be derived from record types. If so, you can use the mapping option:

.option("record_format", "F")
.option("record_length_field", "FIELD_STR")
.option("record_length_map", """{"SEG1":100,"SEG2":200}""") 

where FIELD_STR is the field that determines the record type. When it equals to SEG1 the record length is 100, and so on.

Here are details: https://github.com/AbsaOSS/cobrix?tab=readme-ov-file#variable-length-records-support

[sobhaw]

Ok thx for the solution let me try that. I suppose 100 and 200 will change according to the copybook

[sobhaw]

I tried with 01ABC for record field length it didn't accept a string unsupported function ABC expr syntax error

[sobhaw]

I tried giving the field name also from the copybook

[yruslan]

Please, send all options you are passing to spark-cobol.

[sobhaw]

I cannot send anything outside of my org. So just typing it.
file_start _offset 274, file_end _offset 303
Copybooks , copybooks paths
Variablesize true also checked with recordformat f
Generate record id true
After ur suggestion put a recordlength map 01ABC1 etc