I have an Azure function app that pulls data from snowflake into sharepoint as an xlsx file – this has been running for multiple files for months. I made a change to one of the snowflake tables and the pipeline that the table outputs to is now broken.
Here is the error (sensitive info has been removed/replaced):
2024-08-19T17:31:55Z [Information]
************************************************************************************
Attempting to write output of my_db.my_schema.my_table to sharepoint file file_path_here/target_file.xlsx::testing
************************************************************************************
2024-08-19T17:31:55Z [Information] df sample:
-- sample data removed
[5 rows x 9 columns]
2024-08-19T17:31:56Z [Information] Looking up drive MY_DRIVE Files
2024-08-19T17:31:56Z [Information] Attempting to retrieve file information for file_path_here/target_file.xlsx::testing
2024-08-19T17:31:56Z [Information] Looking up drive MY_DRIVE Files
2024-08-19T17:31:56Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/root:/file_path_here/target_file.xlsx
2024-08-19T17:31:56Z [Information] API call successful!
2024-08-19T17:31:56Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/worksheets
2024-08-19T17:31:56Z [Information] API call successful!
2024-08-19T17:31:56Z [Information] Now listing tables
2024-08-19T17:31:56Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/worksheets/worksheet_id/tables
2024-08-19T17:31:57Z [Information] API call successful!
2024-08-19T17:31:57Z [Information] Now deleting existing tables
2024-08-19T17:31:57Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/tables/table_id_old
2024-08-19T17:31:57Z [Information] API call successful!
2024-08-19T17:31:57Z [Information] Now writing table to sheet testing
2024-08-19T17:31:57Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/worksheets/testing/tables/add
2024-08-19T17:31:58Z [Information] API call successful!
2024-08-19T17:31:58Z [Information] Applying table name:CustomerForecasttest
2024-08-19T17:31:58Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/worksheets/testing/tables/table_id_new
2024-08-19T17:31:59Z [Information] API call successful!
2024-08-19T17:31:59Z [Information] Now writing records to sheet testing
2024-08-19T17:32:00Z [Information] Making API call https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/worksheets/testing/tables/table_id_new/rows/add
2024-08-19T17:32:00Z [Error] API call failed. Attempted to send post command to https://graph.microsoft.com/v1.0/drives/drive_id_here/items/8J398J98DJ39J39JD39JJ9/workbook/worksheets/testing/tables/table_id_new/rows/add but received error code 404
2024-08-19T17:32:00Z [Error] {'error': {'code': 'ResourceNotFound',
'innerError': {'client-request-id': '',
'date': '2024-08-19T17:32:00',
'request-id': ''},
'message': 'Invalid version: error'}}
2024-08-19T17:32:00Z [Error]
******************************************************
Failed download of snowflake object: my_db.my_schema.my_table to sharepoint file MY_DRIVE file_path_here/target_file.xlsx::testing (download_group "daily").
Data successfully collected from Snowflake and transformed.
Failure due to an error writing data to sharepoint.
******************************************************
Traceback (most recent call last):
File "/home/site/wwwroot/my_pipelines/download_snowflake_to_sharepoint.py", line 313, in loop_and_download
sharepoint_conn.write_df_to_sharepoint_excel(df,folderpath=sf_table['sharepoint_folder_name']
File "/home/site/wwwroot/my_pipelines/my_datastores/my_sharepoint.py", line 628, in write_df_to_sharepoint_excel
load_result=self.__query_ms_graph_api(url,type='post',body=payload)
File "/home/site/wwwroot/my_pipelines/my_datastores/my_sharepoint.py", line 147, in __query_ms_graph_api
response.raise_for_status()
File "/home/site/wwwroot/.python_packages/lib/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://graph.microsoft.com/error/error.html?aspxerrorpath=/x/_layouts/xlrestinternal.aspx
2024-08-19T17:32:00Z [Error]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Failed to download table: my_db.my_schema.my_table.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This still works for other files/tables. I know it’s reading the table fine as it outputs the table sample. I also know it’s able to read the file in sharepoint or create it if it doesn’t exist as it’s either last modified recently or just newly created, along with the tab/sheet.
In the tab/sheet, it’s just an empty dataframe (with headers). No clue what is going on – any help is appreciated.