Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
E
enstools-compression
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
w2w
enstools-compression
Commits
19e577f8
Commit
19e577f8
authored
1 year ago
by
Oriol Tintó
Browse files
Options
Downloads
Plain Diff
Merge branch 'main' into development
parents
a914d2be
391c796c
No related branches found
No related tags found
2 merge requests
!18
Several changes including chunking and expansion of examples.
,
!17
Development
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
examples/notebooks/compress_dataset_without_enstools_write.ipynb
+3
-4
3 additions, 4 deletions
...s/notebooks/compress_dataset_without_enstools_write.ipynb
with
3 additions
and
4 deletions
examples/notebooks/compress_dataset_without_enstools_write.ipynb
+
3
−
4
View file @
19e577f8
...
...
@@ -41,9 +41,8 @@
"outputs": [],
"source": [
"import xarray as xr\n",
"import sys\n",
"from pathlib import Path\n",
"from enstools.encoding.api import
Filter
Encoding
ForXarray
"
"from enstools.encoding.api import
Dataset
Encoding"
]
},
{
...
...
@@ -109,7 +108,7 @@
"metadata": {},
"outputs": [],
"source": [
"encoding =
Filter
Encoding
ForXarray
(dataset=dataset, compression=compression)"
"encoding =
Dataset
Encoding(dataset=dataset, compression=compression)"
]
},
{
...
...
@@ -177,4 +176,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}
\ No newline at end of file
%% Cell type:markdown id:2b61cda9-cd2d-4487-b656-93374fa406d6 tags:
# Compress a Dataset without using enstools.io.write
An example on how to use the analyzer to analyze a
**dataset**
and use the results to store it in a compressed file
%% Cell type:markdown id:bc1750af-27da-4ec1-a9a4-9b9646532a6f tags:
### Imports
%% Cell type:code id:d00551a0-599d-4362-b0dd-9861f4919c11 tags:
```
python
import
xarray
as
xr
import
sys
from
pathlib
import
Path
from
enstools.encoding.api
import
Filter
Encoding
ForXarray
from
enstools.encoding.api
import
Dataset
Encoding
```
%% Cell type:markdown id:ce6fd1e9-8e01-425e-9f47-dc86d35b502e tags:
### Download some data
%% Cell type:code id:9aff74be-8f44-40f1-966b-6c23a33f748f tags:
```
python
dataset_name
=
"
air_temperature
"
dataset
=
xr
.
tutorial
.
open_dataset
(
dataset_name
)
dataset
```
%% Cell type:markdown id:ed6e459d-7009-441a-887b-3f5b0acd583f tags:
### Define the compression specifications using the proper compression string format
%% Cell type:code id:d57c590d-1dae-4286-99ce-1a2592863626 tags:
```
python
compression
=
'
lossy,sz,abs,0.14
'
```
%% Cell type:markdown id:8f6dd18c-1385-4108-8add-fad34ee13d90 tags:
### Use enstools.encoding to get the proper encodings for xarray.
%% Cell type:code id:c9979157-a86e-4daf-8aec-3be88eb33432 tags:
```
python
encoding
=
Filter
Encoding
ForXarray
(
dataset
=
dataset
,
compression
=
compression
)
encoding
=
Dataset
Encoding
(
dataset
=
dataset
,
compression
=
compression
)
```
%% Cell type:markdown id:1ec9dc55-5c3f-4ab2-a95f-a2db104ace21 tags:
### Use Xarray to store a compressed netCDF
%% Cell type:code id:a022c4ef-e697-42c1-a73e-86c3769635ba tags:
```
python
# Define a path for the temporary file
file_path
=
Path
(
"
tmp.nc
"
)
```
%% Cell type:code id:0e8f6b3c-2af9-47ea-a644-1f85718d0972 tags:
```
python
# Save dataset to a netcdf file
# Please note that to use the HDF5 filters it is mandatory to use the h5netcdf engine.
dataset
.
to_netcdf
(
file_path
,
engine
=
"
h5netcdf
"
,
encoding
=
encoding
)
```
%% Cell type:code id:35925c92-fbcb-4249-8f8b-b9dc53be9f2d tags:
```
python
# Remove temporary file
if
file_path
.
exists
():
file_path
.
unlink
()
```
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment