Skip to content
Snippets Groups Projects
Commit 112df62f authored by Christoph.Fischer's avatar Christoph.Fischer
Browse files

renamed technique to strategy

parent 127a61d0
No related branches found
No related tags found
No related merge requests found
Showing
with 50 additions and 50 deletions
......@@ -35,13 +35,13 @@ enstools repo serve as base and have been adapted and extended.
Additionally, depending on the used feature identification strategies, additional packages may be required. // TODO
# Usage: Applying existing techniques
# Usage: Applying existing strategies
Here is a usage example, if you want to apply existing techniques in the code base to your data set.
Here is a usage example, if you want to apply existing strategies in the code base to your data set.
First, we need some imports, namely the
* `FeaturePipeline`, which executes the identification pipeline
* `IdentificationTemplate`, this is the identification technique, edit this accordingly
* `TrackingTemplate`, this is the tracking technique, edit this accordingly
* `IdentificationTemplate`, this is the identification strategy, edit this accordingly
* `TrackingTemplate`, this is the tracking strategy, edit this accordingly
* `template_pb2`, this is the on run auto-generated protobuf python file from the set description. Use the one that matches your detection strategy. They are named *_pb2, where * is the name of the identification module.
-> TODO: should not really need to set the template here, is specific to identification strategy!
......@@ -82,19 +82,19 @@ The output data set and description can be saved:
pipeline.save_result(description_type='json', description_path=..., dataset_path=...)
Some of the identification techniques we provide include:
Some of the identification strategies we provide include:
- `african_easterly_waves`: Identify AEWs based on an approach similar to [https://doi.org/10.1002/gdj3.40](Belanger et al. (2016))
- `overlap_example`: Simple starting point to identify objects which should later be tracked via overlap. It creates a new field and writes `i` at positions where object with ID `i` has been identified.
- `pv_streamer`: Identify PV anomalies in 2D (streamers) or 3D, see [https://doi.org/10.5194/gmd-2021-424](Fischer et al. (2022))
- `template` is the starting template for use. If you want to identify areas and track them via overlap, you can use `overlap_example` instead.
Some of the tracking techniques we provide include:
Some of the tracking strategies we provide include:
- `african_easterly_waves`: Tracking of AEWs, by comparing location of line strings.
- `overlap_tracking`: General overlap tracking. It takes the name of the `DataArray` as parameter, ideally one where the values represents the object's id at the location. It works well together with the `overlap_example` identification.
- `template_object_compare`: Template for tracking, where the tracking strategy is solely based on pairwise comparison of object descriptions from consecutive timesteps.
- `template`: Template for a fallback tracking technique which requires more complex heuristics than above mentioned ones.
- `template`: Template for a fallback tracking strategy which requires more complex heuristics than above mentioned ones.
# Usage: Adding techniques
# Usage: Adding strategies
We provide some template files, which we recommend as a starting point for your own identification strategy. If you want to add your own identification (and tracking) strategy to the framework, you need to:
- Copy over the template folder and rename it and the files accordingly. If you implement a tracking method, which relys on pairwise comparison of objects from consecutive timesteps, you can use the `template_object_compare`
......@@ -103,7 +103,7 @@ We provide some template files, which we recommend as a starting point for your
- In the `identification.py` (`tracking.py`), implement your identification (tracking) strategy. See the template again for a useful example. There are a few methods:
- `__init__` gets called from the run script, so the user can set parameters for the algorithm here.
- `precompute` is called once for the entire data set. The data set can be altered here (temporally and spatially). Also if the strategy should return an additional field (`DataArray`), it should be initialized here as shown in the template.
- In `identify` goes your identification technique. This method is called in parallel, and should return a list of objects. See the template and the docstrings for more information. It returns the provided subset (which can be modified in terms of values), and a list of objects. New (empty) objects can be obtained using `o = self.get_new_object(id=obj_id)`, returning an object `o` with the set ID `o.id` and the object properties defined via the protobuf description at `o.properties`.
- In `identify` goes your identification strategy. This method is called in parallel, and should return a list of objects. See the template and the docstrings for more information. It returns the provided subset (which can be modified in terms of values), and a list of objects. New (empty) objects can be obtained using `o = self.get_new_object(id=obj_id)`, returning an object `o` with the set ID `o.id` and the object properties defined via the protobuf description at `o.properties`.
- `postprocess` is called once for the entire data set after identification. The data set and the object description can be changed here.
* TODO tracking
......
Identification
==============
Some of the identification techniques we provide include:
Some of the identification strategies we provide include:
.. toctree::
:maxdepth: 1
......
......@@ -11,7 +11,7 @@ We provide some template files, which we recommend as a starting point for your
- In the `identification.py` (`tracking.py`), implement your identification (tracking) strategy. See the template again for a useful example. There are a few methods:
- `__init__` gets called from the run script, so the user can set parameters for the algorithm here.
- `precompute` is called once for the entire data set. The data set can be altered here (temporally and spatially). Also if the strategy should return an additional field (`DataArray`), it should be initialized here as shown in the template.
- In `identify` goes your identification technique. This method is called in parallel, and should return a list of objects. See the template and the docstrings for more information. It returns the provided subset (which can be modified in terms of values), and a list of objects. New (empty) objects can be obtained using `o = self.get_new_object(id=obj_id)`, returning an object `o` with the set ID `o.id` and the object properties defined via the protobuf description at `o.properties`.
- In `identify` goes your identification strategy. This method is called in parallel, and should return a list of objects. See the template and the docstrings for more information. It returns the provided subset (which can be modified in terms of values), and a list of objects. New (empty) objects can be obtained using `o = self.get_new_object(id=obj_id)`, returning an object `o` with the set ID `o.id` and the object properties defined via the protobuf description at `o.properties`.
- `postprocess` is called once for the entire data set after identification. The data set and the object description can be changed here.
......
......@@ -35,12 +35,12 @@ TODO
Quickstart
----------
Here is a usage example, if you want to apply existing techniques in the code base to your data set.
Here is a usage example, if you want to apply existing strategies in the code base to your data set.
First, we need some imports, namely the
- :py:class:`enstools.feature.pipeline.FeaturePipeline`, which executes the identification pipeline
- :py:class:`enstools.feature.identification.template.IdentificationTemplate`, this is the identification technique, edit this accordingly
- :py:class:`enstools.feature.tracking.template.TrackingTemplate`, this is the tracking technique, edit this accordingly
- :py:class:`enstools.feature.identification.template.IdentificationTemplate`, this is the identification strategy, edit this accordingly
- :py:class:`enstools.feature.tracking.template.TrackingTemplate`, this is the tracking strategy, edit this accordingly
- :py:class:`enstools.feature._proto_gen.template_pb2`, this is the on run auto-generated protobuf python file from the set description. Use the one that matches your detection strategy. They are named \*_pb2, where \* is the name of the identification module.
.. warning::
......
Tracking
========
Some of the tracking techniques we provide include:
Some of the tracking strategies we provide include:
.. toctree::
:maxdepth: 1
......
Custom Tracking
===============
Template for a fallback tracking technique which requires more complex heuristics than above mentioned ones.
Template for a fallback tracking strategy which requires more complex heuristics than above mentioned ones.
Template class
--------------
......
# init file for identification
from .identification import IdentificationTechnique
\ No newline at end of file
from .identification import IdentificationStrategy
\ No newline at end of file
from enstools.feature.identification import IdentificationTechnique
from enstools.feature.identification import IdentificationStrategy
import xarray as xr
import numpy as np
import os, sys
......@@ -14,7 +14,7 @@ import threading
from skimage.draw import line
class AEWIdentification(IdentificationTechnique):
class AEWIdentification(IdentificationStrategy):
def __init__(self, wt_out_file=False, wt_traj_dir=None, cv='cv', year_summer=None, month=None, **kwargs):
"""
......
......@@ -10,10 +10,10 @@ from itertools import product
from enstools.feature.util.data_utils import get_split_dimensions, get_time_split_dimension
class IdentificationTechnique(ABC):
class IdentificationStrategy(ABC):
"""
Base abstract class for new feature identification algorithms. Need to implement abstract methods precompute,
identify and postprocess. An identification technique gets called for a dataset. The dataset is split along
identify and postprocess. An identification strategy gets called for a dataset. The dataset is split along
dimensions to parallelize the identification. The abstract method identify() is called for each spatial block of the
dataset (2D, 3D), for example for each timestamp and for each ensemble member, and if processing_mode is set to
'2d' also for each level. Beforehand, precompute() is called once, here one-time precomputations can be done.
......@@ -124,8 +124,8 @@ class IdentificationTechnique(ABC):
# build meta data for protobuf structure
if not hasattr(self, 'pb_reference'):
print("The protobuf type has not been set in the ID technique.")
print("Check the templates, the IdentificationTechnique.__init__() needs to set the pb2 type.")
print("The protobuf type has not been set in the ID strategy.")
print("Check the templates, the IdentificationStrategy.__init__() needs to set the pb2 type.")
exit(1)
self.pb_dataset = self.pb_reference.DatasetDescription()
......
......@@ -11,7 +11,7 @@ from enstools.feature.util.graph import DataGraph
# set the pb_reference to the compiled pb2.py file (see proto_gen directory)
pipeline = FeaturePipeline(overlap_example_pb2, processing_mode='3d')
# change this to an identification technique that actually does something: existing one or implement your own
# change this to an identification strategy that actually does something: existing one or implement your own
i_strat = OverlapIdentificationExample() # set the Identification strategy
t_strat = OverlapTracking(field_name='identified_areas') # set the tracking strategy
......
from ..identification import IdentificationTechnique
from ..identification import IdentificationStrategy
import xarray as xr
from random import randrange
import datetime
class OverlapIdentificationExample(IdentificationTechnique):
class OverlapIdentificationExample(IdentificationStrategy):
def __init__(self, some_parameter='', **kwargs):
# Constructor. Called from example_template.py, parameters can be passed and set here.
......
from enstools.feature.identification.pv_streamer.processing import *
from enstools.feature.identification import IdentificationTechnique
from enstools.feature.identification import IdentificationStrategy
import xarray as xr
import numpy as np
import copy
......@@ -7,7 +7,7 @@ from enstools.feature.identification.pv_streamer.object_desc import get_object_d
from enstools.feature.identification.pv_streamer.projection_cdo import project_latlon_to_stereo, project_stereo_to_latlon
class PVIdentification(IdentificationTechnique):
class PVIdentification(IdentificationStrategy):
def __init__(self, unit='pv', mode_2d_layer=None, theta_range=None, extract_containing=None, centroid_lat_thr=None,
out_type='stereo', **kwargs):
......
from ..identification import IdentificationTechnique
from ..identification import IdentificationStrategy
from enstools.feature.identification.pv_streamer.data_util import get_pv_field_name
import xarray as xr
......@@ -7,7 +7,7 @@ import numpy as np
from shutil import copyfile
# wrapper calling Wernli Sprenger 2007 identification written in Fortran.
class PVWernliSprenger2007(IdentificationTechnique):
class PVWernliSprenger2007(IdentificationStrategy):
def __init__(self, unit='pv', level=0, **kwargs):
# put custom args
......
......@@ -11,7 +11,7 @@ from enstools.feature.util.graph import DataGraph
# set the pb_reference to the compiled pb2.py file (see proto_gen directory)
pipeline = FeaturePipeline(template_pb2, processing_mode='2d')
# change this to an identification technique that actually does something: existing one or implement your own
# change this to an identification strategy that actually does something: existing one or implement your own
i_strat = IdentificationTemplate(some_parameter='foo') # set the Identification strategy
t_strat = TrackingCompareTemplate() # set the tracking strategy
......@@ -32,7 +32,7 @@ for trackable_set in od.sets:
# generate single tracks from tracked data
# returns list of tracks, also gets added to object description.
# Also if apply_filter, keep_track() gets called for every track, a method of TrackingTechnique, which can be overwritten.
# Also if apply_filter, keep_track() gets called for every track, a method of TrackingStrategy, which can be overwritten.
print("GENERATE TRACKS...")
g.generate_tracks(apply_filter=True)
tracks = g.set_desc.tracks
......
from ..identification import IdentificationTechnique
from ..identification import IdentificationStrategy
import xarray as xr
from random import randrange
class IdentificationTemplate(IdentificationTechnique):
class IdentificationTemplate(IdentificationStrategy):
def __init__(self, some_parameter='', **kwargs):
# Constructor. Called from example_template.py, parameters can be passed and set here.
......
from ..identification import IdentificationTechnique
from ..identification import IdentificationStrategy
from .processing import mask_to_proto, detection_double_thresh
import operator
......@@ -6,7 +6,7 @@ import numpy as np
import xarray as xr
class DoubleThresholdIdentification(IdentificationTechnique):
class DoubleThresholdIdentification(IdentificationStrategy):
def __init__(self, field, outer_threshold, inner_threshold, comparison_operator,
processing_mode="2d", compress=True, **kwargs):
......
......@@ -7,7 +7,7 @@ from os.path import expanduser
# set the pb_reference to the compiled pb2.py file (see proto_gen directory)
pipeline = FeaturePipeline(threshold_pb2, processing_mode='3d')
# change this to an identification technique that actually does something: existing one or implement your own
# change this to an identification strategy that actually does something: existing one or implement your own
path = expanduser("~") + '/PhD/enstools-feature/enstools/data/foo.nc' # ERA5/vietnam/t0_glob.nc' # set data path(s) here
pipeline.set_data_path(path)
......
from enstools.feature.identification import IdentificationTechnique
from enstools.feature.tracking import TrackingTechnique
from enstools.feature.identification import IdentificationStrategy
from enstools.feature.tracking import TrackingStrategy
from enstools.feature.util.enstools_utils import get_vertical_dim
from datetime import datetime
......@@ -32,26 +32,26 @@ class FeaturePipeline:
self.pb_reference = proto_ref
def set_identification_strategy(self, strategy: IdentificationTechnique):
def set_identification_strategy(self, strategy: IdentificationStrategy):
"""
Set the strategy to use for the identification.
Parameters
----------
strategy : IdentificationTechnique
strategy : IdentificationStrategy
The identification strategy to use in the pipeline.
"""
self.id_tech = strategy
self.id_tech.pb_reference = self.pb_reference
self.id_tech.processing_mode = self.processing_mode
def set_tracking_strategy(self, strategy: TrackingTechnique):
def set_tracking_strategy(self, strategy: TrackingStrategy):
"""
Set the strategy to use for the tracking.
Parameters
----------
strategy : TrackingTechnique | None
strategy : TrackingStrategy | None
The tracking strategy to use in the pipeline. Set to `None` or
don't invoke this method at all if no tracking should be carried
out.
......
from .tracking import TrackingTechnique
from .tracking import TrackingStrategy
from .object_compare_tracking import ObjectComparisonTracking
from .tracking import TrackingTechnique
from .tracking import TrackingStrategy
from abc import ABC, abstractmethod
from datetime import datetime
from dask import delayed, compute
......@@ -7,10 +7,10 @@ import xarray as xr
from enstools.feature.util.data_utils import pb_str_to_datetime, SplitDimension
class ObjectComparisonTracking(TrackingTechnique):
class ObjectComparisonTracking(TrackingStrategy):
"""
Implementation of a tracking technique which can track objects by a simple pairwise comparison of their feature
descriptions. This acts as an abstract class for comparison tracking techniques.
Implementation of a tracking strategy which can track objects by a simple pairwise comparison of their feature
descriptions. This acts as an abstract class for comparison tracking strategies.
It implements the track() method and provides an abstract correspond() method with gives a binary answer if two
objects of (consecutive) timesteps are the same. Objects do not have neccessarily be associated with consecutive
timesteps. Using set_max_delta_compare() the user can ser a maximum timedelta.
......@@ -50,8 +50,8 @@ class ObjectComparisonTracking(TrackingTechnique):
def track(self, tracking_set, subset: xr.Dataset):
"""
Implementation of track() for tracking techniques which are based on pairwise comparisons of objects. Using
this tracking technique, the correspond() method has to implement a boolean function which returns True if
Implementation of track() for tracking strategies which are based on pairwise comparisons of objects. Using
this tracking strategy, the correspond() method has to implement a boolean function which returns True if
the given object pair of consecutive time steps should be considered as the same object.
Parameters
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment