Issue with LONG and INT columns when extracting from Oracle
See original GitHub issueHello,
I am extracting data from oracle and generating avro schema based on column types from cx_Oracle. I have the following:
` if defaultType == cx_Oracle.NUMBER:
type = {
"name": name,
"type": ["null", {"name": name, "type": "long"}]
}
avro_schema.append(type)
` The main issue is that output_type_handler in cx_oracle puts LONG and INT under NUMBER datatype (or at least that’s how dba had the tables setup). When trying to generate the avro file with the schema and I encounter an INT column, I get the following error:
ValueError: 3453453453532423323L (type <type 'long'>) do not match ['null', {'type': 'long'}]
So i tried to convert all int and long to decimal:
` if defaultType == cx_Oracle.NUMBER:
type = {
"name": name,
"type": ["null", {"name": name, "type": "bytes", "logicalType": "decimal", "precision": 38, "scale": 0}]
}
avro_schema.append(type)
`
and i get error (i also tried replacing bytes with long) and same issue:
with long:
ValueError: 3453453453532423323L (type <type 'long'>) do not match [{'logicalType': 'decimal', 'scale': 0, 'type': 'long', 'precision': 38}, 'null']
with bytes:
ValueError: 3453232446456443 (type <type 'int'>) do not match [{'logicalType': 'decimal', 'scale': 0, 'type': 'bytes', 'precision': 38}, 'null']
Please let me know if there is any way around this.
I am using python==2.7.12 fastavro==0.21.5 cx-Oracle==6.2
Thank you.
Issue Analytics
- State:
- Created 5 years ago
- Comments:13 (2 by maintainers)
I’m going to close this. I don’t have a way to reproduce it and my suspicion is that they are using the C-API to create their
NUMBER
such that it claims to betype <int>
but doesn’t pass theisinstance
test.@oyamin Somewhere in your code where you have one of these
cx_Oracle.NUMBER
objects could you add the following prints and let me know what they return?