Document Search and Retrieval

Documents can be searched for via meta data, listed out, and then finally retrieved, either from the Kurtosys system directly, or from another remote source.

Searching for and Listing Documents

Searching for documents is simple. You can specify a query to match on meta data, specifying sorting, as well as paging through results. To search for documents, post the following body to the below end point.

Parameter Value
End Point https://api.fundpress.io/documents/searchDocuments
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "type": "CLSS",
    "search": []
}

This should return all documents. Depending on how many documents you have, you may find that the response is too big. In this case, you may want to use paging. To do so you can add a limit and start to the query.

{
    "type": "CLSS",
    "search": [],
    "limit": 100,
    "start": 0
}

In many cases you then may want to sort the list accordingly, perhaps by a tag indicating the fund or share class name. To do that, add a sort block. A sort block contains a key, which targets a particular piece of meta data, and a direction, which can either be ASC or DESC.

{
    "type": "CLSS",
    "search": [],
    "sort": { "key": "fund_name", "direction": "ASC" },
    "limit": 100,
    "start": 0
}

Of course, you can also search! To do so, fill in the empty search block with an array of search terms. Each search term contains a property, which should match a meta data property in the system. matchtype can either be MATCH, for an exact match, or LIKE, for an in-string match. Finally, values is an array of values that the defined property should match on. Note that if you provide more than one value, any documents that match any of the provided values will be returned. So in the below example, both KIID and Factsheet documents will be returned.

{
    "type": "CLSS",
    "search": [
        {
            "property": "document_type",
            "matchtype": "MATCH",
            "values": ["KIID", "Factsheet"]
        }
    ],
    "sort": { "key": "document_type", "direction": "ASC" },
    "limit": 100,
    "start": 0
}

Searching for documents that fall within a specific date range can be achieved by specifying the matchType as RANGE and providing exactly 2 dates in the values array. The first and second values are used as the lower and upper bounds respectively.

{
    "type": "CLSS",
    "search": [
        {
            "property": "document_date",
            "matchtype": "RANGE",
            "values": ["2017-01-01T00:00:00Z", "2017-12-31T23:59:59.999Z"]
        }
    ],
    "limit": 100,
    "start": 0
}

It is also possible to filter by the created or lastModified dates by adding a "meta": false to the search object.

{
    "type": "CLSS",
    "search": [
        {
            "property": "lastModified",
            "matchtype": "RANGE",
            "meta": false,
            "values": ["2017-01-01T00:00:00Z", "2017-12-31T23:59:59.999Z"]
        }
    ],
    "sort": { "key": "document_type", "direction": "ASC" },
    "limit": 100,
    "start": 0
}

If you provide multiple search terms, only items that match both terms will be returned. So in the example below, only items marked with their document_type as KIID and with an isin of GB123123123 will be returned.

{
    "type": "CLSS",
    "search": [
        {
            "property": "document_type",
            "matchtype": "MATCH",
            "values": ["KIID"]
        },
        {
            "property": "isin",
            "matchtype": "MATCH",
            "values": ["GB123123123"]
        }
    ],
    "sort": { "key": "document_type", "direction": "ASC" },
    "limit": 100,
    "start": 0
}

If a null search term is provided, documents that don't have the specified meta data property will be returned. The null search term is only supported when using the MATCH match type.

So in the example below, items will be returned that DONT have an active meta data property as well as items that have an active meta data property that is set to false.

{
    "type": "CLSS",
    "search": [
        {
            "property": "active",
            "matchtype": "MATCH",
            "values": ["false", null]
        }
    ],
    "sort": { "key": "document_type", "direction": "ASC" },
    "limit": 100,
    "start": 0
}

Finally, you can also search for documents unsing their filename. This works in conjunction with the rest of the search filters:

Retrieving Documents

Once a document has been located, the file behind it can be downloaded in a few different ways - via a post, a get, and as a single file or multiple files bundled into a zip.

To retrieve a single document, pass the unique clientCode that identifies the document. Note that you can find the clientCode on each document returned from /searchDocument or attached to a fund or share class in /searchEntity.

If the client has created a "pardotconfig" Client Configuration and that config is enabled=true then the retrieving of any document will call out to Pardot to log the document's retrieval provided the document request has the correct Pardot visitorid cookie (i.e. visitor_id1756).

Parameter Value
End Point https://api.fundpress.io/documents/retrieveDocument
Headers X-KSYS-TOKEN
Content Type application/json or query string for GET
HTTP Method POST or GET
{
    "clientCode": "asdfasdf"
}

Finally, for singular files you can request them to be delivered inline, for instance if you want to put them into an <img> tag or an <iframe>. To do this, add inline=true to the query string.

Retrieving Docs as a Zip

Documents can also be retrieved as a zip file. In this instance, we simply pass a list of the unique codes for the document to the service end point and it will retrieve and zip them before sending back to the caller.

Parameter Value
End Point https://api.fundpress.io/documents/retrieveDocumentZip
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "documentCodes": ["doc1", "doc2", "doc3"]
}

Retrieving Docs By Tags

Documents can also be retrieved using tag values. This allows documents to be tagged with things like a legacy URL and be retrieved accordingly without setting up explicit redirects. Supports the provision of one meta property name and then multiple values. If more than one document matches, a zip will be returned.

Parameter Value
End Point https://api.fundpress.io/documents/retrieveDocumentByTag
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "property": "redirect_url",
    "values": ["https://www.kurtosys.com/docs/uploads/FACTSHEET-KS1319692791-de-DE.pdf"]
}

Retrieving Docs By File Name or Meta Type

Documents can be retrieved using a file name or meta data code. If there are multiple documents that match then a zip file will be returned containing the documents.

Parameter Value
End Point https://api.fundpress.io/documents/retrieve/{authToken}/{option}/{value}
HTTP Method GET
Arguments Description
authToken The authentication token
option Either a metaCode or the string filename
-metaCode Option to match on document meta property
-filename Option to match on filename
value When using filename the value will be the name of the uploaded file, if using a metaCode the value will be the value of the property
-metaCode The value of the meta property to match on
-filename The name of the file

Examples

File name examples

Metacode examples

No matching documents

If there are no matching documents a response code of 400 will be returned.

Example response body:

{
    "error": "No documents found."
}

Searching For Documents by Fund, Share Class or Account Properties

It is possible to search for documents by their related fund or share class properties. This is useful as it allows you to search on a tag that is not directly applied to the document, but may be applied to the fund or share class it relates to.

In order to enable this feature a configuration needs to be created in the system under the client configuration section. To do this via the API you can invoke the following end point.

Parameter Value
End Point https://api.fundpress.io/config/addClientConfiguration
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST

A sample configuration would like the one below.

{
    "code": "document_to_entity_search_conf",
    "config": {
        "document_to_entity_search_configuration": [
            {
                "type": "CLSS",
                "property": "is_primary",
                "joinSpecs": [
                    {
                        "propertyToJoinFrom": "fund_code",
                        "metaToJoinTo": {
                            "value": "fund_code"
                        }
                    },
                    {
                        "propertyToJoinFrom": "isin",
                        "metaToJoinTo": {
                            "value": "isin"
                        }
                    },
                    {
                        "propertyToJoinFrom": "umbrella",
                        "metaToJoinTo": {
                            "value": "umbrella"
                        }
                    }
                ]
            }
        ]
    }
}

You can create as many rules as you like in the document_to_entity_search_configuration array. Each rule tells the system about a field you want to be able to use for searching against documents, but that exists on the fund or share class entity. For each you must specify a type, either CLSS, FUND or ACCT, and a property, which must be a pre-existing fund or share class property code. By doing this you will now be able to search on the specified property.

In the following joinSpecs area you then specify which properties are used to find documents. The reason for this is that there may be a property of a share class that you want to search by, for instance in the above case the is_primary field which indicates if a share class is the primary share class of a fund. However, the documents themselves do not neccessarily have an is_primary field on them. However, they do have tags such as isin or fund_code or umbrella. The joinSpecs therefore tell the system which fields to use to find the documents.

In the above example, the system would find share classes by the is_primary field, harvest their fund_code, isin, and umbrella values, and then find any documents that have either of those values on them, in an or fashion.

Finally, the metaToJoinTo key is the name of the meta value you are targetting, as there is no guarantee the fields on the fund and class have the same name. You also have an ability to use an inferred meta field as the target of the join. You can use the key "isInferred": true in the metaToJoinTo to do this.

However, it should be clear that by setting up these values alone does not invoke a search, it simply lays the ground work for doing an entity linked search. To do the search we come back to the document search service itself.

Parameter Value
End Point https://api.fundpress.io/documents/searchDocuments
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "type": "CLSS",
    "search": [],
    "entitySearch": [
        {
            "property": "is_primary",
            "values": ["true"],
            "entityType": "CLSS"
        }
    ]
}

You can see in the above example we've invoked a search on the is_primary field, asking for all classes where that value is true. So long as there is a rule setup for the is_primary field, the system will execute the search as described above.

There are some features and restrictions with the feature:

Retrieve possible filters

This would allow you to list all possible property value options for document meta data properties. Take the example below, it will query all the possible property/tag values for documents with the tags of document_type, ccy, invalid_code, it will also further filter down if you supply it with a search clause. Keep in mind, if you have document entitlements enabled, it will filter your results based on your entitlement configuration.

Parameter Value
End Point https://api.fundpress.io/documents/getFilters
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "codes": ["document_type", "ccy", "invalid_code"],
    "search": [
        {
            "property": "document_type",
            "values": ["KIID", "Factsheet"],
            "matchtype": "MATCH"
        }
    ]
}
Property Required Value
codes true An array of tags/codes/documentMetaCodes
search false An array of search clauses

The above query will scan all your documents and return a listing of the various meta values for the document_type, ccy and invalid_code property.

If a property is not valid document meta then it will be returned with an empty array.

{
    "ccy": [
        {
            "label": "na",
            "value": "GBP"
        },
        {
            "label": "na",
            "value": ["ZAR", "USD"]
        }
    ],
    "document_type": [
        {
            "label": "na",
            "value": "Factsheet"
        },
        {
            "label": "na",
            "value": "KIID"
        }
    ],
    "invalid_code": []
}

By passing through the search property in the payload,

{
    "codes": ["document_type", "ccy", "invalid_code"],
    "search": [
        {
            "property": "document_type",
            "values": ["Factsheet", "KIID"],
            "matchtype": "MATCH"
        }
    ]
}

you can filter down the codes even more

{
    "ccy": [],
    "document_type": [
        {
            "label": "na",
            "value": "Factsheet"
        },
        {
            "label": "na",
            "value": "KIID"
        }
    ],
    "invalid_code": []
}

Document Facets

It is possible to return facets based on a search term. The supported facet types are document meta, document filename and entity properties as defined by the document to entity search configuration. Results are grouped by property and limited to 10 results per propertyValues.

Parameter Value
End Point https://api.fundpress.io/documents/getFacets
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "searchTerm": "fact"
}

Example response

[
    {
        "property": "Document Type",
        "propertyValues": [
            {
                "propertyValue": "Factsheet",
                "property": "Document Type",
                "code": "document_type",
                "facetType": "META"
            }
        ]
    },
    {
        "property": "File Name",
        "propertyValues": [
            {
                "propertyValue": "FACTSHEET.pdf",
                "property": "File Name",
                "code": "filename",
                "facetType": "DOCUMENT_PROPERTY"
            }
        ]
    }
]

Document Entitlements

A user with the role 'DocumentAdmin' will bypass all entitlement filters and be able to list, retrieve or update all documents on the respective client.

User Document Entitlement Schemes

This will set the provided users document entitlement scheme to the one schemeId provided. Either the userId or externalUserIdentifier can be used to identify the user to update.

Parameter Value
End Point https://api.fundpress.io/documents/upsertUserDocumentEntitlementScheme
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "userId": 20,
    "documentEntitlementSchemeId": 1
}

Or

{
    "externalUserIdentifier": "K99282ss92",
    "documentEntitlementSchemeId": 1
}
Property Required Description
userId or externalUserIdentifier true The user whose entitlement scheme needs to be updated
documentEntitlementSchemeId true The document entitlement scheme id

Upsert User Document Entitlement

Parameter Value
End Point https://api.fundpress.io/documents/upsertUserDocumentEntitlements
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "userId": 20,
    "documentEntitlementSchemeId": 1,
    "documentEntitlements": [
        {
            "tag": "document_type",
            "value": "factsheet",
            "endDate": "2017-12-15"
        },
        {
            "tag": "document_type",
            "value": "image",
            "startDate": "2017-11-11",
            "endDate": "2017-12-15"
        },
        {
            "tag": "isin",
            "value": "f1232",
            "startDate": "2017-11-11",
            "endDate": "2017-12-14"
        }
    ]
}
Property Required Description
userId true The user whose entitlements needs to be updated
documentEntitlementSchemeId false Optionally you can update the user's entitlement scheme by passing this through
documentEntitlements true the entitlement tags and dates

List User Document Entitlements

Parameter Value
End Point https://api.fundpress.io/documents/listUserDocumentEntitlements
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "userId": 20
}

Delete User Document Entitlements

Parameter Value
End Point https://api.fundpress.io/documents/deleteUserDocumentEntitlements
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "userId": 20,
    "documentEntitlements": [
        {
            "tag": "document type",
            "value": "isin"
        },
        {
            "tag": "document type",
            "value": "isin",
            "startDate": "2017-11-11"
        },
        {
            "tag": "document type",
            "value": "other",
            "startDate": "2017-11-11"
        }
    ]
}

Bulk Change of User Document Entitlement Schemes

Bulk Add Entitlements

This endpoint adds document entitlements to multiple users at a time.

A user can only have one Entitlement Scheme at a time. If any users are already assigned to an entitlement scheme that is different to the one in the request then the process exists early and returns the invalid users.

Parameter Value
End Point https://api.fundpress.io/documents/bulkAddDocumentEntitlements
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "userNames": ["usernameOne", "usernameTwo", "usernameThree"],
    "documentEntitlementSchemeName": "Marketing Entitlements",
    "documentEntitlements": [
        {
            "tag": "document_type",
            "value": "factsheet",
            "endDate": "2017-12-15"
        },
        {
            "tag": "document_type",
            "value": "image",
            "startDate": "2017-11-11",
            "endDate": "2017-12-15"
        },
        {
            "tag": "isin",
            "value": "f1232",
            "startDate": "2017-11-11",
            "endDate": "2017-12-14"
        }
    ]
}

This can also be called with the following instead of the userNames array:

{
    "externalUserIdentifiers": ["identifierOne", "identifierTwo", "identifierThree"]
}

When there are users that are already assigned to a different Entitlement Scheme and userNames were sent in the request then the response is as follows:

{
    "invalidUsers": [105, 107, 119],
    "userNames": ["usernameFive", "usernameSever", "usernameOneOneNine"],
    "externalUserIdentifiers": []
}

When there are users that are already assigned to a different Entitlement Scheme and externalUserIdentifiers were sent in the request then the response is as follows:

{
    "invalidUsers": [105, 107, 119],
    "userNames": [],
    "externalUserIdentifiers": ["exterIdXX8", "exterIdYZ93", "exterIdAbb52"]
}

Bulk Replace Entitlements

Parameter Value
End Point https://api.fundpress.io/documents/bulkReplaceDocumentEntitlements
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "userNames": ["usernameOne", "usernameTwo", "usernameThree"],
    "documentEntitlementSchemeName": "Marketing Entitlements",
    "documentEntitlements": [
        {
            "tag": "document_type",
            "value": "factsheet",
            "endDate": "2017-12-15"
        },
        {
            "tag": "document_type",
            "value": "image",
            "startDate": "2017-11-11",
            "endDate": "2017-12-15"
        },
        {
            "tag": "isin",
            "value": "f1232",
            "startDate": "2017-11-11",
            "endDate": "2017-12-14"
        }
    ]
}

This can also be called with the following instead of the userNames array:

{
    "externalUserIdentifiers": ["identifierOne", "identifierTwo", "identifierThree"]
}

Document Entitlement Schemes

Upsert

To upsert a document entitlement scheme, use the following end-point. The name is unique per client, using the same name will result in updating the scheme instead of creating a new one.

Parameter Value
End Point https://api.fundpress.io/documents/upsertDocumentEntitlement
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "name": "abc",
    "tags": ["blue", "green", "red"]
}

Or, for updating:

{
    "documentEntitlementSchemeId": 123,
    "name": "New ABC",
    "tags": ["blue", "green"]
}
Property Required Description
documentEntitlementSchemeId false The id of the Document Entitlement Scheme, if updating one.
name true The name of the entitlement scheme, must be unique per client
tags true A list of tags associated with the scheme

List

Get a list of the document entitlement schemes.

Parameter Value
End Point https://api.fundpress.io/documents/listDocumentEntitlements
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST

Delete

To delete a document entitlement scheme, make use of the following endpoint.

Parameter Value
End Point https://api.fundpress.io/documents/deleteDocumentEntitlement
Headers X-KSYS-TOKEN
Content Type application/json
HTTP Method POST
{
    "documentEntitlementSchemeId": 20
}
Property Required Description
documentEntitlementSchemeId true The id of the item that needs to be deleted

Saved Searches

Once you have performed a search, you have the ability to save this search for future use. You must give this search a name and choose to be notified on the search. If you opt for notifications, these can be sent out on a Daily or Weekly occurence.

List Saved Searches

Parameter Value
End Point https://api.fundpress.io/documents/getSavedSearches
Headers X-KSYS-TOKEN X-KSYS-APP
Content Type application/json
HTTP Method POST
{
    "savedSearchName": "KIID"
}

Example response

{
    "total": 2,
    "values": [
        {
            "savedSearchId": 1,
            "savedSearchName": "TBC KIIDs",
            "notificationFrequency": "Daily",
            "created": "2017-09-08T00:00:00.000Z",
            "lastModified": "2017-12-08T10:37:56.123Z",
            "search": {
                "type": "CLSS",
                "search": [
                    {
                        "matchtype": "MATCH",
                        "property": "document_type",
                        "values": ["KIID"]
                    },
                    {
                        "matchtype": "MATCH",
                        "property": "isin",
                        "values": ["TBC"]
                    }
                ],
                "sort": {
                    "key": "isin",
                    "direction": "DESC"
                }
            }
        },
        {
            "savedSearchId": 2,
            "savedSearchName": "RW KIIDs",
            "created": "2017-10-08T00:00:00.000Z",
            "lastModified": "2017-12-08T10:37:56.123Z",
            "search": {
                "type": "CLSS",
                "search": [
                    {
                        "matchtype": "MATCH",
                        "property": "document_type",
                        "values": ["KIID"]
                    },
                    {
                        "matchtype": "MATCH",
                        "property": "isin",
                        "values": ["RWCEFF000001"]
                    }
                ],
                "sort": {
                    "key": "isin",
                    "direction": "DESC"
                }
            }
        }
    ]
}

The response does not return a limit as this returns all values.

Upsert Saved Search

Parameter Value
End Point https://api.fundpress.io/documents/upsertSavedSearch
Headers X-KSYS-TOKEN X-KSYS-APP
Content Type application/json
HTTP Method POST

Creation Call

For creation, no savedSearchId is passed. The X-KSYS-APP is used to determine the application that this saved search belongs to.

{
    "savedSearchName": "RW KIIDs",
    "notificationFrequency": "weekly",
    "search": {
        "type": "CLSS",
        "search": [
            {
                "matchtype": "MATCH",
                "property": "document_type",
                "values": ["KIID"]
            }
        ],
        "sort": {
            "key": "filename",
            "direction": "ASC"
        },
        "filenames": ["KIID-KS0273642768-de-DE.pdf"]
    }
}

Example response

{
    "savedSearchName": "RW KIIDs",
    "notificationFrequency": "weekly",
    "search": {
        "type": "CLSS",
        "search": [
            {
                "matchtype": "MATCH",
                "property": "document_type",
                "values": ["KIID"]
            }
        ],
        "sort": {
            "key": "filename",
            "direction": "ASC"
        },
        "filenames": ["KIID-KS0273642768-de-DE.pdf"]
    }
}

Update Call

For update, the savedSearchId is passed as well as anything that should change.

{
    "savedSearchId": 1,
    "savedSearchName": "RW KIIDs renamed",
    "notificationFrequency": "daily"
}

Example response

{
    "savedSearchId": 1,
    "savedSearchName": "RW KIIDs renamed",
    "notificationFrequency": "daily",
    "search": {
        "type": "CLSS",
        "search": [
            {
                "matchtype": "MATCH",
                "property": "document_type",
                "values": ["KIID"]
            }
        ],
        "sort": {
            "key": "filename",
            "direction": "ASC"
        },
        "filenames": ["KIID-KS0273642768-de-DE.pdf"]
    }
}

Delete Saved Search

Parameter Value
End Point https://api.fundpress.io/documents/deleteSavedSearch/
Headers X-KSYS-TOKEN X-KSYS-APP
Content Type application/json
HTTP Method POST

This will only work if you are the creator of the saved search. Returns a 204.

{
    "savedSearchId": "1234"
}

Document Filename Check

Parameter Value
End Point https://api.fundpress.io/documents/documentExists
Headers X-KSYS-TOKEN X-KSYS-APP
Content Type application/json
HTTP Method POST

Takes a list of document filenames. Will check if any of them already exist for the given client.

{
    "documentFilenames": ["KS123456780.pdf", "KS123456781.pdf", "KS123456782.pdf", "KS123456783.pdf"]
}
Property Type Required Description
documentFilenames STRING ARRAY FALSE An array of document filenames with extensions

Will return an empty list if the documents do not already exist, and a list containing duplicates if they match an already existing document.

No Existing Documents

Status: 200

{
    "existingDocuments": []
}

Existing Documents

Status: 200

{
    "existingDocuments": ["KS123456780.pdf"]
}