Skip to main content

Senseforce

Availability:
Available Airbyte CloudAvailable Airbyte OSSAvailable PyAirbyte
Support Level:
Marketplace
Connector Version:
0.2.4(Last updated 16 days ago)
Sync Success Rate:
Usage Rate:

This page guides you through the process of setting up the Senseforce source connector.

The Senseforce Airbyte connector allows to export custom datasets built bei Senseforce users. Follow these steps to configure a dataset which can be exported with the Airbyte connector:

  1. Create a new, empty dataset as documented here
  2. Add at least the following columns (these columns are Senseforce system columns and available for all of your custom data models/event schemas):
    1. Metadata -> Timestamp
    2. Metadata -> Thing
    3. Metadata -> Id
  3. Add any other column of your event schemas you want to export
  4. Enter a descriptive Name and a Description and save the dataset
  5. Note the ID of the dataset (the GUID at the end of the URL path of your dataset in your browser URL bar)

Tip: For most exports it is recommended to have the Timestamp column in first place. The Airbyte connector automatically orders in ascending direction. If the Timestamp column is not in the first position, incremental syncs might not work properly.

IMPORTANT: The Timestamp, Thing and Id column are mandatory for the Connector to work as intended. While it still works without eg. the "Id", functionality might be impaired if one of these 3 columns is missing. Make sure to not rename these columns - keep them at their default names.

  1. Log into your Airbyte Cloud or Airbyte Open Source account.

  2. Click Sources and then click + New source.

  3. On the Set up the source page, select Senseforce from the Source type dropdown.

  4. Enter a name for your source.

  5. For API Access Token, enter your Senseforce API Access Token.

  6. For Senseforce backend URL, enter your Senseforce Backend URL.

  7. For Dataset ID, enter your Senseforce Dataset ID.

    We recommend creating an api access token specifically for Airbyte to control which resources Airbyte can access. For good operations, we recommend to create a separate Airbyte User as well as a separate Senseforce Airbyte Group. Share the dataset with this group and grant Dataset Read, Event Schema Read and Machine Master Data Read permissions.

  8. For The first day (in UTC) when to read data from, enter the day in YYYY-MM-DD format. The data added on and after this day will be replicated.

  9. Click Set up source.

The Senseforce source connector supports the following sync modes:

  • Full Refresh
  • Incremental

NOTE: The Senseforce Airbyte connector uses the Timestamp column to determine, which data were already read. Data inserted AFTER a finished sync, with timestamps less than already synced ones, are not considered for the next sync anymore. If this behavior does not fit your use case, follow the next section

  1. Rename your "Timestamp" column to "Timestamp_data"
  2. Add the Metadata -> Inserted column to your dataset.
  3. Move the newly added "Inserted" column to position 1.
  4. Rename the "Inserted" column to "Timestamp".

Now the inserted timestamp will be used for creating the Airbyte cursor. Note that this method results in slower syncs, as the Senseforce queries to generate the Datasets are slower than if you use the data timestamp.

The Senseforce source connector supports the following streams:

FeatureSupported?(Yes/No)Notes
Full Refresh SyncYes
Incremental SyncYes

Senseforce utilizes an undocumented rate limit which - under normal use - should not be triggered, even with huge datasets. Create an issue if you see any rate limit issues that are not automatically retried successfully.

Expand to review
VersionDatePull RequestSubject
0.2.42024-11-0448166Update dependencies
0.2.32024-10-2947765Update dependencies
0.2.22024-10-2847567Update dependencies
0.2.12024-08-1644196Bump source-declarative-manifest version
0.2.02024-08-1444073Refactor connector to manifest-only format
0.1.152024-08-1243865Update dependencies
0.1.142024-08-1043704Update dependencies
0.1.132024-08-0343274Update dependencies
0.1.122024-07-2742763Update dependencies
0.1.112024-07-2042247Update dependencies
0.1.102024-07-1341792Update dependencies
0.1.92024-07-1041402Update dependencies
0.1.82024-07-0941242Update dependencies
0.1.72024-07-0640914Update dependencies
0.1.62024-06-2540380Update dependencies
0.1.52024-06-2240190Update dependencies
0.1.42024-06-1539462Make compatible with builder
0.1.32024-06-0439073[autopull] Upgrade base image to v1.2.1
0.1.22024-05-2038431[autopull] base image + poetry + up_to_date
0.1.12023-02-1322892Specified date formatting in specification
0.1.02022-10-26#18775🎉 New Source: Mailjet SMS API [low-code CDK]
Was this page helpful?