Dev/ftr 10024 convert defc codes to upper #4382
Conversation
|
I'm not sure if I made the changes in the correct spot in the code. Also the ticket mentions there might be additional fields but only specifically mentions DEFC codes, I don't know what else might need to be upper case |
loreleitrimberger
added
the
ready for review
…com/fedspendingtransparency/usaspending-api into dev/ftr-convert-defc-codes-to-upper
…com/fedspendingtransparency/usaspending-api into dev/ftr-convert-defc-codes-to-upper
| disaster_emergency_fund_code = row["disaster_emergency_fund_code"] | ||
| if not disaster_emergency_fund_code: | ||
| return None | ||
| try: | ||
| return DISASTER_EMERGENCY_FUND_CODES[disaster_emergency_fund_code] | ||
| return DISASTER_EMERGENCY_FUND_CODES[disaster_emergency_fund_code.upper()] |
There was a problem hiding this comment.
Stepping through the code more to re-familiarize myself I don't believe you need to make a change here. This method is only used when loading File C. Prior to calling this method a helper method is used to handle uppercasing all strings found in each File C row. See here
.| @@ -230,18 +230,18 @@ def setUp(self): | |||
| prior_year_adjustment | |||
| ) (values | |||
| (1, 1, 1, '1010', 'D', 1111, null, 'x'), | |||
| (2, 1, 1, '1010', 'D', 2222, 'B', 'b'), | |||
| (2, 1, 1, '1010', 'D', 2222, 'b', 'b'), | |||
There was a problem hiding this comment.
The test data you have manipulated here is for loading File B. A good starting point for where you need to make a change is looking here:
| @@ -263,7 +263,7 @@ def setUp(self): | |||
| ) (values | |||
| (1, 1, 1, '1010', 'D', 11111, 111110, -11, -111, null, 'x'), | |||
| (2, 1, 1, '1010', 'D', 22222, 222220, -22, -222, 'B', 'b'), | |||
| (3, 1, 1, '1010', 'D', 33333, 333330, -33, -333, 'L', 'p'), | |||
| (3, 1, 1, '1010', 'D', 33333, 333330, -33, -333, 'l', 'p'), | |||
There was a problem hiding this comment.
If you remove the change to usaspending_api/etl/submission_loader_helpers/disaster_emergency_fund_codes.py I believe this should still pass.
Description:
DEFC codes need to be capitalized because they are case sensitive in USA Spending, but not within Broker where agencies can submit DEFC codes case insensitive. This is to fix the nightly pipeline from failing due to this
Technical details:
Convert DEFC codes to uppercase during the ETL process as well as other text coming over
Requirements for PR merge:
Area for explaining above N/A when needed: