Merge branch 'master' of https://git.scc.kit.edu/sp2668/esm_tutorials

.gitignore

@@ -7,3 +7,4 @@
 **/__pycache__/
 *.lp
 *.slides.html
+tutorial-6/network_data/

requirements.yml

@@ -5,6 +5,7 @@ channels:
 dependencies:
     - jupyter
     - matplotlib
+    - basemap
     - nb_conda
     - numpy
     - scipy

tutorial-2/solution02.ipynb

@@ -766,9 +766,8 @@
   }
  ],
  "metadata": {
-  "celltoolbar": "Slideshow",
   "kernelspec": {
-   "display_name": "Python [default]",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -782,7 +781,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.4"
   },
   "nav_menu": {},
   "toc": {

tutorial-6/sheet06.tex

@@ -90,69 +90,56 @@
  \textbf{\Large Energy System Modelling }\\
  {SS 2018, Karlsruhe Institute of Technology}\\
  {Institute for Automation and Applied Informatics}\\ [1em]
- \textbf{\textsc{\Large Tutorial VI: Clustering of Large Power Systems\\}}
+ \textbf{\textsc{\Large Tutorial VI: Clustering Large Power Systems\\}}
  \small Will be worked on in the exercise session on Wednesday, 18 July 2018.\\[1.5em]
 \end{center}
 
 \vspace{1em}
 
 %=============== ======================================================
-\paragraph{Clustering the European Grid} ~\\
+\paragraph{Problem VI.1 \normalsize (clustering the European grid).}~\\
 %=====================================================================
 
-As seen previously, the integration of renewable energy is cheaper at a large geographical scale and its variability requires a good temporal resolution. Therefore the computational power needed for the optimization of the whole energy system could be too high for servers or super computer.
-
-In this tutorial, we will first compare 2 basic clustering solutions to reduce the grid model size. Then we will observe the impact of clustering on the lowest installed capacity required to feed the demand.
-
-%=============== ======================================================
-\paragraph{Introduction – the European grid.}~\\
-%=====================================================================
-
-the file \textit{base.nc} contains a good representation of the European grid, with the various high voltage AC lines and the DC links.
-
-You can load this file in PyPSA and plot so that the colors of the lines represent the voltage type and level, the width of the lines are proportional to their transmission capacity.
-
-\textbf{Task:} Calculate the order and the size of this graph (you will need to include AC lines and DC links).
-
-To cluster this network, we will use a simplified version of this grid, by considering all AC lines to have the same voltage. It will also group very close buses. Import the simplified version (\textit{elec\char`_s.nc}).
-
-\textbf{Task:} Calculate the resulting size and order.
-
-
-%=============== ======================================================
-\paragraph{Clustering with sklearn.}~\\
-%=====================================================================
-
-Sklearn is a library which allows to cluster data. Due to its simplicity and to the limited goal of this tutorial, we will cluster the simplified network by grouping the buses according to their geographical locations.
-
-\textbf{Task:} Perform various clustering of the simplified network by changing clustering algorithms (Kmeans, SpectralClustering, …) and the number of clusters $k$. \\
-\mbox{\url{http://scikit-learn.org/stable/modules/clustering.html}}
-
-\textbf{Task:} Use the \textit{plot\char`_clusters} function of tutorial.py to observe the outcome of the clustering.
-
-\textbf{Questions:}
-\begin{itemize}
-  \item Are the buses adequately clustered?
-  \item Which algorithm is giving the best outcome?
-  \item What are the limits of using geographically based data clustering on the electrical grid?
-\end{itemize}
-
-%=============== ======================================================
-\paragraph{Clustering and minimum installed capacity.}~\\
-%=====================================================================
-
-The file  \textit{elec\char`_s.nc} provides solar and wind energy hourly profile per bus for year 2013 and an hourly load profile per bus. The Run of River Hydropower is also fixed.
-
-We will now use the functions :\\
-\textit{tutorial.find\char`_kmeans\char`_busmap} and \textit{pypsa.networkclustering.get\char`_clustering\char`_from\char`_busmap} \\
-to cluster the European Grid according to the Kmeans algorithm with weighted values (corresponding here to the maximum load at the bus). These functions aggregate the bus data (load, renewable potential...) to the cluster node and group lines from the initial network to form the edges of the clustered graph.
-
-Our clustered network is still missing some installed capacities for renewable at each bus to be able to run a simulation such as a Linear Optimal Power Flow (LOPF): LOPF could say if our installed capacity of renewable could supply the loads according to the available line transfer capacity.
-
-In order to give a capacity of solar, onshore wind and\char`/or offshore wind to each bus in a simple way, I created a small function \textit{P\char`_nom\char`_re} attributing an installed capacity proportional to load at each bus (the proportionality coefficients for solar w\char`_s, onshore wind w\char`_on and offshore wind w\char`_of could be changed independently). The goal here is not to simulate a real or ideal energy system, but to be more familiar with the tools and to observe if different clustering sizes or clustering groups are having an impact on results.
-
-\textbf{Task:} For a graph with a small number of cluster (10-25), determine minimum proportionality coefficients w\char`_s, w\char`_on and w\char`_of to have a successful LOPF at a specific day and hour (observe the message \textit{INFO:pypsa.opf:Optimization successful} or \textit{ERROR:pypsa.opf:Optimisation failed}).
-
-\textbf{Questions:} If you try the same coefficients with a higher number of cluster (100-200), is the LOPF still successful. If not, can you find an explanation?
+In the preceding tutorials we have seen that the integration of renewable energy is cheaper when they are integrated over a large geographical area and that its variability requires a fine temporal resolution. In consequence, however, the computational power needed for the optimization of the whole energy system is immense and could be prohibitively high. In this tutorial, we will first compare 2 basic clustering solutions to reduce the grid model size and thereby alleviate the computational burden. Then we will observe the impact of clustering on the lowest installed capacity required to feed the demand.
+
+\begin{figure}[h]
+	\centering
+	\includegraphics[width=\linewidth,trim={3cm 5cm 3cm 5cm},clip]{europeangrid}
+	
+	\caption{The European power grid.}
+	\label{test}
+\end{figure}
+
+\begin{enumerate}[(a)]
+	\item The file \texttt{network\_data/base.nc} contains a detailed representation of the European power grid, including the various high voltage AC lines and the DC links. 
+	\begin{enumerate}[(i)]
+		\item Load the file into the provided Jupyter notebook and plot it. The \textit{colours} of the lines represent the voltage type and level, whereas the \textit{width }of the lines are proportional to their transmission capacity. 
+		\item Calculate the order and the size of this graph. You will need to include AC lines and DC links.
+		\item To cluster this network, we will use a simplified version of this grid, by considering all AC lines to have the same voltage. The simplified version also groups buses which are very close to each other.
+		
+		Import the simplified version (\texttt{network\_data/elec\_s.nc}).
+		\item Calculate the order and the size of the simplified graph.
+	\end{enumerate}
+
+	\item We will now turn to clustering the European power network using \textit{Sklearn}. Sklearn is a library which allows the clustering of data. For the sake of exposition, we will cluster the network simply by grouping buses according to their geographical locations. There are, of course, more elaborate approaches which, however, exceed the scope of the tutorial.
+	\begin{enumerate}[(i)]
+		\item Perform various clustering configurations of the network by altering the clustering algorithms and the number of clusters k. Find the available clustering algorithms at \url{http://scikit-learn.org/stable/modules/clustering.html}.
+		\item Use the \texttt{plot\_clusters} function of the package \texttt{tutorial.py} to observe the outcome of the clustering.
+		\item Are the buses adequately clustered? Which algorithm is giving the best outcome? What are the limits of using geographically based data clustering on the electrical grid? Comment on your findings!
+	\end{enumerate}
+
+	\item We will now look at the interaction between the clustering configuration and minimum installed capacity in the network. The network file \texttt{network\_data/elec\_s.nc} also provides solar and wind energy hourly profile per bus for the year 2013 as well as an hourly load profile per bus. The run-of-river hydropower is also fixed.
+	
+	We will make use of the functions \texttt{tutorial.find\_kmeans\_busmap} and \texttt{pypsa.} \texttt{networkclustering.get\_clustering\_from\_busmap} to cluster the European power grid according to the K-Means algorithm, with weights corresponding to the maximum load at the respective bus. These functions aggregate the bus data (i.e.\ load, renewable energy potential, etc.) to the cluster nodes and group lines from the initial network to form the edges of the clustered graph.
+	
+	The clustered network is missing some installed capacities for renewables at each bus to be able to run a linear optimal power flow (LOPF). In order to set a capacity of solar, onshore wind and\/or offshore wind to each bus in a simple way, we use the function \texttt{tutorial.P\_nom\_re(network)} to attribute an installed capacity proportional to the load at each bus. The proportionality coefficients for solar \texttt{w\_s}, onshore wind \texttt{w\_on} and offshore wind \texttt{w\_of} could be changed independently. Keep in might, this is just for illustrative purposes probably not very accurate in reality.
+	
+	\begin{enumerate}[(i)]
+		\item For a graph with a small number of clusters (e.g.\ between 10 and 25), determine minimum proportionality coefficients \texttt{w\_s}, \texttt{w\_on} and \texttt{w\_of} to have a feasible solution for the LOPF at a specific day and hour.
+		
+		\item If you try the same coefficients with a higher number of clusters (e.g.\ between 100 and 200), is the LOPF still feasible. If not, could you find an explanation?
+	\end{enumerate}
+
+\end{enumerate}
 
 \end{document}

tutorial-6/tutorial6utilities.py → tutorial-6/tutorial.py

@@ -8,11 +8,11 @@ import matplotlib as mpl
 import matplotlib.pyplot as plt
 import matplotlib.lines as mlines
 from sklearn.cluster import KMeans, SpectralClustering
+from mpl_toolkits.basemap import Basemap
 
 
-### FUNCTIONS
-
 def plot_clusters(network, y_pred,X,lim_fig="None"):
+    
     colors_choice=['b', 'g', 'r', 'c', 'y', 'violet','purple','k','chartreuse','indianred','pink','orange','gray','yellow','springgreen','indigo','brown','silver','aqua']*(int(max(y_pred)/19)+1)
     colors_list = [ colors_choice[x] for x in y_pred]
 
@@ -32,10 +32,10 @@ def plot_clusters(network, y_pred,X,lim_fig="None"):
     plt.show()
 
 def plot_network(network,option="AC_DC",basemap="no"):
+
     fig, ax = plt.subplots()
     fig.set_size_inches(15,15)
 
-
     if basemap=="yes":
         long_min=-10
         long_max=30
@@ -74,8 +74,9 @@ def plot_network(network,option="AC_DC",basemap="no"):
         violet_line=mlines.Line2D([], [], color='violet',label='DC')
 
         plt.legend(handles=[blue_line,green_line,orange_line,red_line,violet_line])
-        
+
 def find_kmeans_busmap(n_clusters, n, n_weightings, **kwargs):
+
     kmeans = KMeans(init='k-means++', n_clusters=n_clusters, ** kwargs)
 
     kmeans.fit(n[["x","y"]].values)
@@ -88,7 +89,7 @@ def weighting(network):
     load = (network.loads_t.p_set.mean()
                    .groupby(network.loads.bus).sum()
                    .reindex(network.buses.index, fill_value=0.))
-    
+
     return load
 
 
@@ -103,6 +104,7 @@ def P_nom_re(network,w_s=2.5,w_onw=0.2,w_ofw=0.1):
     network.generators.loc[network.generators['carrier'] == 'offwind',"p_nom"]=network.generators.loc[network.generators['carrier'] == 'offwind',"weight"]*w_ofw
 
 def lopf_d_h(network,day,hour):
+
     network.set_snapshots(pd.DatetimeIndex(start=hour.format(day), end=hour.format(day), freq='H'))
 
     # solve linear optimal power flow
@@ -110,6 +112,7 @@ def lopf_d_h(network,day,hour):
                      solver_name='glpk')
 
 def plot_line_loading(network):
+
     fig, ax = plt.subplots()
     ax.set_ylim([35,72])
     fig.set_size_inches(15,15)
@@ -129,13 +132,14 @@ def plot_line_loading(network):
     Z = [[0,0],[max(abs(loading)),0]]
     CS3 = plt.contourf(Z,  cmap=plt.cm.jet)
     plt.colorbar(fraction=0.01, pad=0.01)
-    
+
 def plot_bus_status(network):
+
     fig, ax = plt.subplots()
     ax.set_ylim([35,72])
     fig.set_size_inches(15,15)
 
-    bus_status= pd.DataFrame({'status': network.loads_t.p_set.values[0]*-1, 
+    bus_status= pd.DataFrame({'status': network.loads_t.p_set.values[0]*-1,
                           'pmax':network.loads_t.p_set.values[0]*0,
                           'p':network.loads_t.p_set.values[0]*0,
                           'ponpmax':network.loads_t.p_set.values[0]*0})
@@ -152,9 +156,9 @@ def plot_bus_status(network):
             bus_status.ponpmax.loc[i]=bus_status.p.loc[i]/bus_status.pmax.loc[i]
         except:
             print(i)
-            
+
     bus_color=(bus_status.status/p_nc_25.loads_t.p_set.values[0]).values
-    
+
     #print(loading_dc)
     #print(pd.concat(dict(Line=abs(loading),Link=abs(loading_dc))))
     #network.plot(line_colors=abs(loading),line_cmap=plt.cm.jet,title="Line loading",
@@ -166,8 +170,9 @@ def plot_bus_status(network):
     Z = [[min(bus_color),0],[max(bus_color),0]]
     CS3 = plt.contourf(Z,  cmap=plt.cm.jet)
     plt.colorbar(fraction=0.01, pad=0.01)
-    
+
 def plot_line_loading_bus_status(network):
+
     fig, ax = plt.subplots()
     ax.set_ylim([35,72])
     fig.set_size_inches(15,15)
@@ -177,8 +182,8 @@ def plot_line_loading_bus_status(network):
     loading_dc = network.links_t.p0.loc[network.snapshots[0]]/network.links.p_nom/0.7
     loading_dc=loading_dc.fillna(0)
     loading_dc=loading_dc.append(pd.Series(1))
-    
-    bus_status= pd.DataFrame({'status': network.loads_t.p_set.values[0]*-1, 
+
+    bus_status= pd.DataFrame({'status': network.loads_t.p_set.values[0]*-1,
                           'pmax':network.loads_t.p_set.values[0]*0,
                           'p':network.loads_t.p_set.values[0]*0,
                           'ponpmax':network.loads_t.p_set.values[0]*0})
@@ -194,12 +199,12 @@ def plot_line_loading_bus_status(network):
 
             bus_status.ponpmax.loc[i]=bus_status.p.loc[i]/bus_status.pmax.loc[i]
         except:
-            print(i)
-            
+            print("Error at {}".format(i))
+
     bus_color=(bus_status.status/network.loads_t.p_set.values[0]).values
     bus_color[bus_color>1.5]=1.5
-    
-    
+
+
     #print(loading_dc)
     #print(pd.concat(dict(Line=abs(loading),Link=abs(loading_dc))))
     #network.plot(line_colors=abs(loading),line_cmap=plt.cm.jet,title="Line loading",
@@ -213,7 +218,7 @@ def plot_line_loading_bus_status(network):
                  line_widths=pd.concat(dict(Line=network.lines['s_nom']/500,Link=network.links['p_nom']/500)))
     Z = [[0,0],[max(abs(loading)),0]]
     CS3 = plt.contourf(Z,  cmap=plt.cm.jet)
-    plt.colorbar(fraction=0.01, pad=0.06)
+    plt.colorbar(fraction=0.01, pad=0.06, label='Residual Generation divided by Load at Bus')
     Z2 = [[min(bus_color),0],[max(bus_color),0]]
     CS4 = plt.contourf(Z2,  cmap=plt.cm.jet)
-    plt.colorbar(fraction=0.01, pad=0.01)
\ No newline at end of file
+    plt.colorbar(fraction=0.01, pad=0.01, label='Line loading [pu]')