16. Querying Web Services

This notebook describes how to query web services in RAW.

There are two RAW features that help when querying web services: * Support for complex data - web services often produce complex JSON or XML documents; * Caching - caching means web services queries can be cached and reused automatically.

[3]:
%load_ext raw_magic
The raw_magic extension is already loaded. To reload it, use:
  %reload_ext raw_magic

As an illustration, we look at a complex WHO public web service which outputs a complex XML, e.g.:

<GHO xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Created="2021-01-10T15:09:49Z" xsi:schemaLocation="ghodata.xsd">
<QueryParameter Name="format" Value="xml"/>
<QueryParameter Name="baseurl" Value="https://apps.who.int/gho/athena/api/"/>
<QueryParameter Name="filter" Value="COUNTRY:-;REGION:*"/>
<QueryParameter Name="asof" Value="2021-01-10 10:01:26.0"/>
<QueryParameter Name="metadata" Value="active"/>
<QueryParameter Name="target" Value="GHO/WHOSIS_000001"/>
<Copyright>
<Display>(c) World Health Organization</Display>
</Copyright>
<Disclaimer>
<Display>
The information in this database is provided as a service to our users. Any use of information in the web site should be accompanied by an acknowledgment of WHO as the source. The responsibility for the interpretation and use of the material lies with the user. In no event shall the World Health Organization be liable for any damages arising from the use of the information linked to this section.
</Display>
</Disclaimer>
<Metadata>
<Dataset Label="CYCU">
<Display>COUNTRY_YEARLY_CORE_UNITS</Display>
</Dataset>
<Attribute Label="CATEGORY" EntityType="CORE_DIMENSION" Entity="GHO">
<Display>Category</Display>
</Attribute>
...

In fact, querying this data in RAW is as simple as executing a single query. We set cache duration to null, so that data is cached indefinitely.

[4]:
%%rql

READ("https://apps.who.int/gho/athena/api/GHO/WHOSIS_000001?filter=COUNTRY:-;REGION:*", cache := null)
[4]:
@Created@schemaLocationQueryParameterCopyrightDisclaimerMetadataData
@Name@ValueDisplayDisplayDatasetAttributeDimensionObservation
@LabelDisplay@Label@EntityType@EntityDisplay@Label@IsMeasureDisplayCode@FactID@Published@Dataset@EffectiveDate@EndDateDimValue
@Label@DisplaySequence@URLAttrDisplay@Category@Code@NumericDisplay
@CategoryValue
Display
2021-01-10T15:10:13Zghodata.xsdformatxml(c) World Health OrganizationThe information in this database is provided as a service to our users. Any use of information in the web site should be accompanied by an acknowledgment of WHO as the source. The responsibility for the interpretation and use of the material lies with the user. In no event shall the World Health Organization be liable for any damages arising from the use of the information linked to this section.CYCUCOUNTRY_YEARLY_CORE_UNITSCATEGORYCORE_DIMENSIONGHOCategoryGHOTrueIndicatorWHOSIS_00000110https://www.who.int/data/gho/indicator-metadata-registry/imr-details/65DISPLAY_FREsperance de vie a la naissance (ans)Life expectancy at birth (years)25053134TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000160.4672460.5
baseurlhttps://apps.who.int/gho/athena/api/RENDERER_IDCORE_DIMENSIONGHORenderer IDDISPLAY_ESEsperanza de vida al nacer YEAR2015
filterCOUNTRY:-;REGION:*DEFINITION_XMLCORE_DIMENSIONGHODefinition (XML)DEFINITION_XMLhttp://apps.who.int/gho/indicatorregistryservice/publicapiservice.asmx/IndicatorGetAsXml?profileCode=WHO&applicationCode=System&languageAlpha2=en&indicatorId=65SEXMLE
asof2021-01-10 10:01:26.0DISPLAY_FRCORE_DIMENSIONGHOFrançaisCATEGORYSustainable development goalsREGIONAFR
metadataactiveDISPLAY_ESCORE_DIMENSIONGHOEspañolRENDERER_IDRENDER_2PUBLISHSTATEPUBLISHED
targetGHO/WHOSIS_000001PUBLISHSTATEnullPUBLISH STATESPUBLISHED0nullemptyPublished25053150TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000164.6899264.7
YEARnullYear200079998000nullempty2000SEXFMLE
201079897990nullempty2010YEAR2015
201579847985nullempty2015REGIONAFR
201979807981nullempty2019PUBLISHSTATEPUBLISHED
REGIONnullWHO regionAFR10nullemptyAfrica25053170TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000164.4922964.5
AMR20nullemptyAmericasREGIONAFR
SEAR30nullemptySouth-East AsiaSEXBTSX
EUR40nullemptyEuropeYEAR2019
EMR50nullemptyEastern MediterraneanPUBLISHSTATEPUBLISHED
WPR60nullemptyWestern Pacific25053186TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000174.4876274.5
GLOBAL5nullemptyGlobalREGIONAMR
WORLDBANKINCOMEGROUPnullWorld Bank income groupWB_WORLD_INCOME5nullemptyGlobalSEXMLE
SEXnullSexMLE2nullemptyMaleYEAR2019
FMLE3nullemptyFemalePUBLISHSTATEPUBLISHED
BTSX1nullemptyBoth sexes25053202TrueCYCU2020-12-072900-12-31SEXFMLE79.8391479.8
YEAR2019
REGIONAMR
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053218TrueCYCU2020-12-072900-12-31SEXBTSX77.1587377.2
REGIONAMR
GHOWHOSIS_000001
YEAR2019
PUBLISHSTATEPUBLISHED
25053234TrueCYCU2020-12-072900-12-31REGIONSEAR69.8822669.9
GHOWHOSIS_000001
YEAR2019
SEXMLE
PUBLISHSTATEPUBLISHED
25053250TrueCYCU2020-12-072900-12-31SEXFMLE73.1015873.1
GHOWHOSIS_000001
REGIONSEAR
YEAR2019
PUBLISHSTATEPUBLISHED
25053270TrueCYCU2020-12-072900-12-31SEXMLE68.4401768.4
GHOWHOSIS_000001
YEAR2000
REGIONEUR
PUBLISHSTATEPUBLISHED
25053286TrueCYCU2020-12-072900-12-31SEXFMLE76.6444776.6
REGIONEUR
YEAR2000
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053302TrueCYCU2020-12-072900-12-31YEAR200072.5114172.5
SEXBTSX
REGIONEUR
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053318TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000163.5194263.5
SEXMLE
YEAR2000
REGIONEMR
PUBLISHSTATEPUBLISHED
25053334TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED66.6744566.7
REGIONEMR
YEAR2000
GHOWHOSIS_000001
SEXFMLE
25053350TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED65.0168265.0
REGIONEMR
GHOWHOSIS_000001
YEAR2000
SEXBTSX
25053354TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED67.5765867.6
REGIONEMR
SEXBTSX
YEAR2010
GHOWHOSIS_000001
25053370TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED72.5235172.5
GHOWHOSIS_000001
REGIONWPR
SEXMLE
YEAR2010
25053386TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED78.7472678.7
REGIONWPR
SEXFMLE
YEAR2010
GHOWHOSIS_000001
25053402TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED75.4646575.5
GHOWHOSIS_000001
REGIONWPR
YEAR2010
SEXBTSX
25053470TrueCYCU2020-12-072900-12-31REGIONGLOBAL69.7676669.8
YEAR2015
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
SEXMLE
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053486TrueCYCU2020-12-072900-12-31REGIONGLOBAL74.8883174.9
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
GHOWHOSIS_000001
SEXFMLE
YEAR2015
PUBLISHSTATEPUBLISHED
25053502TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000172.2777872.3
YEAR2015
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
SEXBTSX
REGIONGLOBAL
PUBLISHSTATEPUBLISHED
25053130TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED57.6855957.7
SEXMLE
YEAR2010
GHOWHOSIS_000001
REGIONAFR
25053146TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED61.3100361.3
GHOWHOSIS_000001
YEAR2010
SEXFMLE
REGIONAFR
25053162TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED59.4869459.5
GHOWHOSIS_000001
REGIONAFR
SEXBTSX
YEAR2010
25053166TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED62.560662.6
GHOWHOSIS_000001
REGIONAFR
SEXBTSX
YEAR2015
25053182TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED74.051574.1
REGIONAMR
GHOWHOSIS_000001
YEAR2015
SEXMLE
25053198TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED79.519179.5
YEAR2015
GHOWHOSIS_000001
SEXFMLE
REGIONAMR
25053214TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED76.7843676.8
YEAR2015
GHOWHOSIS_000001
SEXBTSX
REGIONAMR
25053230TrueCYCU2020-12-072900-12-31SEXMLE68.7000268.7
YEAR2015
GHOWHOSIS_000001
REGIONSEAR
PUBLISHSTATEPUBLISHED
25053246TrueCYCU2020-12-072900-12-31YEAR201571.7840771.8
SEXFMLE
GHOWHOSIS_000001
REGIONSEAR
PUBLISHSTATEPUBLISHED
25053266TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000171.4435571.4
REGIONSEAR
SEXBTSX
YEAR2019
PUBLISHSTATEPUBLISHED
25053282TrueCYCU2020-12-072900-12-31YEAR201975.0938775.1
REGIONEUR
GHOWHOSIS_000001
SEXMLE
PUBLISHSTATEPUBLISHED
25053298TrueCYCU2020-12-072900-12-31SEXFMLE81.2929281.3
YEAR2019
REGIONEUR
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053314TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000178.2426178.2
YEAR2019
REGIONEUR
SEXBTSX
PUBLISHSTATEPUBLISHED
25053330TrueCYCU2020-12-072900-12-31REGIONEMR68.3112168.3
GHOWHOSIS_000001
SEXMLE
PUBLISHSTATEPUBLISHED
YEAR2019
25053346TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED71.3139671.3
GHOWHOSIS_000001
SEXFMLE
YEAR2019
REGIONEMR
25053366TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED69.796169.8
SEXMLE
YEAR2000
REGIONWPR
GHOWHOSIS_000001
25053382TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED75.3253975.3
GHOWHOSIS_000001
REGIONWPR
YEAR2000
SEXFMLE
25053398TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED72.4222672.4
GHOWHOSIS_000001
REGIONWPR
SEXBTSX
YEAR2000
25053466TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED67.9715768.0
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
SEXMLE
YEAR2010
GHOWHOSIS_000001
REGIONGLOBAL
25053482TrueCYCU2020-12-072900-12-31YEAR201073.2040373.2
REGIONGLOBAL
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
GHOWHOSIS_000001
SEXFMLE
PUBLISHSTATEPUBLISHED
25053498TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000170.5278770.5
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
SEXBTSX
REGIONGLOBAL
YEAR2010
PUBLISHSTATEPUBLISHED
25053126TrueCYCU2020-12-072900-12-31REGIONAFR51.057651.1
YEAR2000
SEXMLE
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053142TrueCYCU2020-12-072900-12-31REGIONAFR54.3095754.3
SEXFMLE
GHOWHOSIS_000001
YEAR2000
PUBLISHSTATEPUBLISHED
25053158TrueCYCU2020-12-072900-12-31REGIONAFR52.6536552.7
SEXBTSX
GHOWHOSIS_000001
YEAR2000
PUBLISHSTATEPUBLISHED
25053178TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000172.529172.5
SEXMLE
YEAR2010
REGIONAMR
PUBLISHSTATEPUBLISHED
25053194TrueCYCU2020-12-072900-12-31SEXFMLE78.3548978.4
REGIONAMR
YEAR2010
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053210TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000175.4371375.4
REGIONAMR
SEXBTSX
YEAR2010
PUBLISHSTATEPUBLISHED
25053226TrueCYCU2020-12-072900-12-31SEXMLE66.3866766.4
REGIONSEAR
GHOWHOSIS_000001
YEAR2010
PUBLISHSTATEPUBLISHED
25053242TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000169.9870870.0
REGIONSEAR
YEAR2010
SEXFMLE
PUBLISHSTATEPUBLISHED
25053262TrueCYCU2020-12-072900-12-31YEAR201570.1972970.2
SEXBTSX
GHOWHOSIS_000001
REGIONSEAR
PUBLISHSTATEPUBLISHED
25053278TrueCYCU2020-12-072900-12-31SEXMLE73.8015873.8
REGIONEUR
GHOWHOSIS_000001
YEAR2015
PUBLISHSTATEPUBLISHED
25053294TrueCYCU2020-12-072900-12-31REGIONEUR80.30880.3
YEAR2015
GHOWHOSIS_000001
SEXFMLE
PUBLISHSTATEPUBLISHED
25053310TrueCYCU2020-12-072900-12-31SEXBTSX77.0896277.1
GHOWHOSIS_000001
YEAR2015
REGIONEUR
PUBLISHSTATEPUBLISHED
25053326TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000166.8100966.8
REGIONEMR
SEXMLE
YEAR2015
PUBLISHSTATEPUBLISHED
25053342TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED70.1388770.1
REGIONEMR
YEAR2015
GHOWHOSIS_000001
SEXFMLE
25053362TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED69.743969.7
YEAR2019
REGIONEMR
GHOWHOSIS_000001
SEXBTSX
25053378TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED74.7610974.8
GHOWHOSIS_000001
SEXMLE
YEAR2019
REGIONWPR
25053394TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED80.8322880.8
REGIONWPR
YEAR2019
SEXFMLE
GHOWHOSIS_000001
25053410TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED77.6632977.7
REGIONWPR
GHOWHOSIS_000001
SEXBTSX
YEAR2019
25053462TrueCYCU2020-12-072900-12-31WORLDBANKINCOMEGROUPWB_WORLD_INCOME64.4190264.4
GHOWHOSIS_000001
REGIONGLOBAL
YEAR2000
SEXMLE
PUBLISHSTATEPUBLISHED
25053478TrueCYCU2020-12-072900-12-31YEAR200069.2701669.3
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
REGIONGLOBAL
GHOWHOSIS_000001
SEXFMLE
PUBLISHSTATEPUBLISHED
25053494TrueCYCU2020-12-072900-12-31WORLDBANKINCOMEGROUPWB_WORLD_INCOME66.794366.8
REGIONGLOBAL
GHOWHOSIS_000001
YEAR2000
SEXBTSX
PUBLISHSTATEPUBLISHED
25053138TrueCYCU2020-12-072900-12-31REGIONAFR62.3687362.4
SEXMLE
GHOWHOSIS_000001
YEAR2019
PUBLISHSTATEPUBLISHED
25053154TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000166.6457566.6
YEAR2019
REGIONAFR
SEXFMLE
PUBLISHSTATEPUBLISHED
25053174TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED71.1317271.1
REGIONAMR
GHOWHOSIS_000001
YEAR2000
SEXMLE
25053190TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED77.0995377.1
SEXFMLE
YEAR2000
REGIONAMR
GHOWHOSIS_000001
25053206TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED74.1214174.1
REGIONAMR
GHOWHOSIS_000001
SEXBTSX
YEAR2000
25053222TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED62.1979262.2
REGIONSEAR
GHOWHOSIS_000001
YEAR2000
SEXMLE
25053238TrueCYCU2020-12-072900-12-31REGIONSEAR64.5771464.6
GHOWHOSIS_000001
YEAR2000
SEXFMLE
PUBLISHSTATEPUBLISHED
25053254TrueCYCU2020-12-072900-12-31YEAR200063.3529663.4
SEXBTSX
REGIONSEAR
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053258TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000168.1141868.1
SEXBTSX
YEAR2010
REGIONSEAR
PUBLISHSTATEPUBLISHED
25053274TrueCYCU2020-12-072900-12-31SEXMLE72.1392472.1
GHOWHOSIS_000001
YEAR2010
REGIONEUR
PUBLISHSTATEPUBLISHED
25053290TrueCYCU2020-12-072900-12-31GHOWHOSIS_00000179.2653479.3
SEXFMLE
REGIONEUR
YEAR2010
PUBLISHSTATEPUBLISHED
25053306TrueCYCU2020-12-072900-12-31YEAR201075.7165975.7
SEXBTSX
GHOWHOSIS_000001
REGIONEUR
PUBLISHSTATEPUBLISHED
25053322TrueCYCU2020-12-072900-12-31SEXMLE66.0338566.0
REGIONEMR
GHOWHOSIS_000001
YEAR2010
PUBLISHSTATEPUBLISHED
25053338TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED69.2731169.3
GHOWHOSIS_000001
YEAR2010
SEXFMLE
REGIONEMR
25053358TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED68.3892868.4
GHOWHOSIS_000001
REGIONEMR
YEAR2015
SEXBTSX
25053374TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED73.9648974.0
SEXMLE
GHOWHOSIS_000001
REGIONWPR
YEAR2015
25053390TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED80.3228680.3
SEXFMLE
GHOWHOSIS_000001
YEAR2015
REGIONWPR
25053406TrueCYCU2020-12-072900-12-31PUBLISHSTATEPUBLISHED76.9830277.0
GHOWHOSIS_000001
REGIONWPR
YEAR2015
SEXBTSX
25053474TrueCYCU2020-12-072900-12-31REGIONGLOBAL70.8476870.8
GHOWHOSIS_000001
SEXMLE
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
YEAR2019
PUBLISHSTATEPUBLISHED
25053490TrueCYCU2020-12-072900-12-31YEAR201975.8722375.9
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
REGIONGLOBAL
SEXFMLE
GHOWHOSIS_000001
PUBLISHSTATEPUBLISHED
25053506TrueCYCU2020-12-072900-12-31REGIONGLOBAL73.3133173.3
GHOWHOSIS_000001
WORLDBANKINCOMEGROUPWB_WORLD_INCOME
YEAR2019
SEXBTSX
PUBLISHSTATEPUBLISHED

We may now do a specific query:

[7]:
%%rql

SELECT `@Name`, `@Value`
FROM READ("https://apps.who.int/gho/athena/api/GHO/WHOSIS_000001?filter=COUNTRY:-;REGION:*", cache := null).QueryParameter
[7]:
@Name@Value
formatxml
baseurlhttps://apps.who.int/gho/athena/api/
filterCOUNTRY:-;REGION:*
asof2021-01-10 10:01:26.0
metadataactive
targetGHO/WHOSIS_000001