38 Commits

Author SHA1 Message Date
Sebastian Lenzlinger
8be3279ace Merge branch 'dev-explore' into 'main'
Milestone Merge

See merge request dbis/lecture-groups/database-systems/2023hs/group-1!3
2023-12-03 22:58:31 +00:00
Sebastian Lenzlinger
9e2709eaaa Merge branch 'main' into 'dev-explore'
# Conflicts:
#   README.md
2023-12-03 22:56:37 +00:00
Sebastian Lenzlinger
49bb3f4e20 Data Integration Milestone Completed.
Refactored ensuring all relevant dirs exist into separate script.
2023-12-03 23:53:36 +01:00
Michel Romancuk
1ab8c9fb2e Update file README.md 2023-12-03 21:57:42 +00:00
Sebastian Lenzlinger
e9b1d82517 FINAL TOUCH: Add functions to get data from api link.
Refactor bash script into python function to ensure portability.
Add sql queries to create "Contemporaneous" db table.
2023-12-03 22:52:20 +01:00
Sebastian Lenzlinger
b79ee792b6 Preliminary: Completed Integration Scripts. Add scripts that load data into a database of choice. Some config is still manual. 2023-12-03 21:40:29 +01:00
Sebastian Lenzlinger
fcfb3f028b Intermediate Commit 2023-12-03 20:46:54 +01:00
Sebastian Lenzlinger
45b4f7b998 Merge remote-tracking branch 'origin/dev-explore' into dev-explore
# Conflicts:
#	src/integrate.py
#	src/setup_tables.sql
2023-12-03 19:27:35 +01:00
Sebastian Lenzlinger
39b28a4b65 Merge remote-tracking branch 'origin/dev-explore' into dev-explore
# Conflicts:
#	src/integrate.py
#	src/setup_tables.sql
2023-12-03 19:27:02 +01:00
Sebastian Lenzlinger
dcf921292b Merge remote-tracking branch 'origin/dev-explore' into dev-explore
# Conflicts:
#	src/integrate.py
#	src/setup_tables.sql
2023-12-03 17:37:07 +01:00
Sebastian Lenzlinger
90a38b1bb5 Drop duplicates in unified miv csv. 2023-12-03 17:36:29 +01:00
Sebastian Lenzlinger
920f725d74 Drop duplicates in unified miv csv. 2023-12-03 17:05:09 +01:00
Sebastian Lenzlinger
c33ca87aaf UNFINNISHED: Script to create csv of integrated and clean datasets. 2023-12-03 16:34:40 +01:00
Sebastian Lenzlinger
1ef7bbe39b Convert MIV coordinate to Int 2023-12-03 13:37:11 +01:00
Sebastian Lenzlinger
94ee3cc3b0 ADD ID to fb. 2023-12-03 13:18:01 +01:00
Sebastian Lenzlinger
65bd9480e7 Load accident file from internet if not present. 2023-12-03 12:35:38 +01:00
Sebastian Lenzlinger
0afab39397 Merge remote-tracking branch 'origin/dev-explore' into dev-explore
# Conflicts:
#	src/data_utils.py
#	src/integrate.py
2023-12-03 12:30:26 +01:00
Sebastian Lenzlinger
ca3450a4de Add logger. 2023-12-03 12:30:15 +01:00
Sebastian Lenzlinger
e4d0484a23 Add logger. 2023-12-03 12:28:32 +01:00
Sebastian Lenzlinger
e73962d8e1 + 2023-12-03 11:48:32 +01:00
Sebastian Lenzlinger
cd0bb639ca Merge remote-tracking branch 'origin/dev-explore' into dev-explore 2023-12-03 11:48:19 +01:00
Sebastian Lenzlinger
793b783dd9 + 2023-12-03 11:48:11 +01:00
Sebastian Lenzlinger
983f85b116 Fix requirements.txt 2023-12-03 11:45:00 +01:00
Sebastian Lenzlinger
250e13acba Fix requirements.txt 2023-12-03 11:44:42 +01:00
Sebastian Lenzlinger
b6715a0ff1 UNFINNISHED: Commit for Syncing Repos 2023-12-03 01:09:32 +01:00
Sebastian Lenzlinger
101243fae8 Add test notebook for debugging on different machine 2023-12-03 00:28:25 +01:00
Sebastian Lenzlinger
e07932a113 UNFINNISHED: Add data integration scripts. 2023-12-02 22:26:13 +01:00
Sebastian Lenzlinger
77bf140efc Add Text File containing relevant datasource urls.
First diary entries written.
Wiki entries on how to setup a python virtual env for the project
2023-12-02 22:10:30 +01:00
Sebastian Lenzlinger
653c3341ce Add Text File containing relevant datasource urls.
First diary entries written.
Wiki entries on how to setup a python virtual env for the project
2023-11-16 18:40:55 +01:00
Sebastian Lenzlinger
8cf5940a4d Add Text File containing relevant datasource urls.
First diary entries written.
Wiki entries on how to setup a python virtual env for the project
2023-11-16 18:04:10 +01:00
Sebastian Lenzlinger
552b49f118 Add Text File containing relevant datasource urls.
First diary entries written.
Wiki entries on how to setup a python virtual env for the project
2023-11-16 17:59:10 +01:00
Sebastian Lenzlinger
f39d3d5963 Merge branch 'dev-explore' into 'main'
Add Text File containing relevant datasource urls

See merge request dbis/lecture-groups/database-systems/2023hs/group-1!2
2023-11-16 15:50:19 +00:00
Sebastian Lenzlinger
b47ee9ea1d Add Text File containing relevant datasource urls 2023-11-16 16:49:10 +01:00
Sebastian Lenzlinger
c70ecbc11e Merge branch 'dev-explore' into 'main'
Dev explore

See merge request dbis/lecture-groups/database-systems/2023hs/group-1!1
2023-11-16 15:25:30 +00:00
Sebastian Lenzlinger
401ce9dd30 Update .gitignore 2023-11-16 15:24:47 +00:00
Sebastian Lenzlinger
b29ce4278c Add Data Exploration Notebook. 2023-11-15 14:04:39 +01:00
Sebastian Lenzlinger
22a72aea6c Add a diary and a wiki document. 2023-11-07 13:46:35 +01:00
Raphael
26e24d1441 Initial Commit with README.md 2023-10-10 18:44:17 +02:00