Fixes/#220

[eosram]

The implementation of the timeseries generator script gave often rise to questions. The script intended by me as a quick workaround is not very human readable anymore. With #220 as a starting point I tried to reimplement it in python.

[IlkaCu]

I am wondering if this conflicts with the changes @MarlonSchlemminger implemented in #233.

[MarlonSchlemminger]

Yeah, #233 and #227 as well would need to be included in the python script. Just checked on my local database and at least for p_max_pu the bug is gone after 233.

[eosram]

Thank you for your feedback @MarlonSchlemminger , @IlkaCu ! If I got this right p_max_pu have to be set based on ren_feedin_by_gen_id for German renewable sources and ren_feedin_foreign for all other sources. Offshore for foreign countries is handled differently and is based on ego_renewable_feedin, ego_neighbours_offshore_point. Is this correct @MarlonSchlemminger ? Do I miss something? If so, I guess, that's where I could start.

[eosram]

There might be a problem in the current dev.

data_processing/dataprocessing/sql_snippets/ego_dp_powerflow_timeseries_generator.sql

Lines 68 to 96 in dc096ef

	DROP materialized view IF EXISTS model_draft.ren_feedin_by_gen_id;
	CREATE materialized view model_draft.ren_feedin_by_gen_id
	AS
	SELECT
	gen.generator_id, feedin.feedin
	FROM
	(SELECT
	aggr_id AS generator_id,
	w_id,
	power_class,
	source
	FROM
	model_draft.ego_supply_pf_generator_single
	WHERE source IN (12, 13, 17)
	AND scn_name = 'Status Quo'
	GROUP BY aggr_id, w_id, power_class, source) AS gen,
	(SELECT
	w_id,
	power_class,
	CASE
	WHEN source LIKE '%%solar%%' THEN 12
	WHEN source LIKE '%%wind_onshore%%' THEN 13
	WHEN source LIKE '%%wind_offshore%%' THEN 17
	END AS source,
	feedin
	FROM model_draft.ego_renewable_feedin) AS feedin
	WHERE gen.source = feedin.source
	AND gen.w_id = feedin.w_id
	AND gen.power_class = feedin.power_class;

The query will return multiple duplicate generator ids with corresponding, but different feedins based on the different power_class, w_id (?).

data_processing/dataprocessing/sql_snippets/ego_dp_powerflow_timeseries_generator.sql

Lines 114 to 118 in dc096ef

	-- set p_max_pu as timeseries from ego_renewable_feedin
	UPDATE model_draft.ego_grid_pf_hv_generator_pq_set A
	SET p_max_pu = feedin.feedin
	FROM model_draft.ren_feedin_by_gen_id AS feedin
	WHERE A.generator_id = feedin.generator_id;

The update is based on generator_id alone which will assign one of the multiple corresponding feedins provided by the previously constructed view, if I'm not mistaken.

This is related to #245. If you think, it's best, I will reopen that pull reqest!

I tried to handle it in f4765af afccca9. With regard to this pull request there is unfortunately still some/a lot work to do and I'm struggling with the different ids. Maybe I can ask you about it @IlkaCu

[MarlonSchlemminger]

In my local view, there is only one entry per generator_id. There are some rows without a generator_id where I don't know why they exist, but they shouldn't have any effect because they will not be used during the assignment of the feedins. I can't test it on the oedb, because the view doesn't exist there. Where did you get the problem, on your local database? Did you run the new version of ego_dp_powerflow_assignment_generator.sql before?

[eosram]

Deleted my previous comment to avoid further confusion. Thanks @MarlonSchlemminger, I assumed the aggr_id would be a unique combination of scn_name, bus, source, but I now see, that it also takes into account the w_id and power_class.

[eosram]

Hmm, does ego_neighbours_offshore_point have the right permissions set? I do not know why, but I'm not able to make a backup of it.

[wolfbunke]

right, I just changed it ✅

[eosram]

Still missing in this PR to address bug #220 is the assignment of offshore wind feedin data for neighbouring countries, which I can maybe add on Friday at the earliest. Also, and that's maybe a dealbreaker, the PR does rely on an updated ego.io https://github.com/openego/ego.io/tree/fixes/dataprocessing-%23220/egoio/db_tables. While the release is running changing dependencies might contradict version control in some way. But I guess, it's also important, that this PR after finalization still has to be reviewed, which might take some time too. I guess, it depends on whether there is a next release.

[eosram]

The pull request is open for review again : ). Forgot to mention that.

[wolfbunke]

ok, I will check it later

[eosram]

I just now realized another reason, why storage was commented out up until now and why stay this way.