This document outlines the options for linking or retrieving the Carerix Datasource files.
The datasource files are made available on our FTP server and/or in an Amazon S3 bucket.
1. Testing the Connection with netcat (FTP server)
1. Testing the Connection with netcat (FTP server)
Operating System: Linux and macOS
Tested on macOS 15.2
Requirements: The IP address used to perform the netcat test must be on our whitelist.
Explicit FTPS
% nc -vvv datasource.carerix.net 21
Connection to datasource.carerix.net port 21 [tcp/ftp] succeeded!
220-FileZilla Server 1.9.3
220 Please visit https://filezilla-project.org/
2. FileZilla client GUI (FTP server)
2. FileZilla client GUI (FTP server)
Operating System: Linux, Windows, and macOS
Tested on macOS 15.2 with FileZilla Client 3.68.1 Apple Silicon (arm64)
Requirements:
FileZilla client
FTP username
FTP password
IP address used with FileZilla client must be on our whitelist
If an outbound firewall is used, TCP ports 21 and 21000–21010 must be allowed for IP address 3.121.45.103
3. WinScp client command line interface (FTP server)
3. WinScp client command line interface (FTP server)
Operating System: Windows
Tested on Windows 11 with WinScp 6.1.1
Requirements:
WinScp (Typical install)
FTP username
FTP password
IP address used with WinScp must be on our whitelist
If an outbound firewall is used, TCP ports 21 and 21000–21010 must be allowed for IP address 3.121.45.103
Open command prompt (cmd.exe)
"C:\Program Files (x86)\WinSCP\WinSCP.com"
winscp> open ftpes://carerix.user1:xxxxxx@datasource.carerix.net:21
Connecting to datasource.carerix.net ...
TLS connection established. Waiting for welcome message...
Connected
Starting the session...
Session started.
Active session: [1] xxxxxx.user1@datasource.carerix.net
winscp> ls crmatch*
D--------- 0 0 ..
---------- 0 23245 Aug 20 1:22:28 2023 Cragency.csv
...
winscp> get crcompany.csv c:\temp\
crcompany.csv | 36260 KB | 2131.9 KB/s | binary | 100%
winscp> exit
als commando in een batch file
C:\Program Files (x86)\WinSCP\WinSCP.com /command ^
"ftpes://carerix.user1:xxxxxx@datasource.carerix.net:21" ^
"get *.csv c:\temp\" ^
"exit"
4. Cyberduck GUI (S3 bucket)
4. Cyberduck GUI (S3 bucket)
Operating system: Linux, Windows en macOS
Tested on macOS 15.2 with Cyberduck client 9.1.0
Requirements
AWS S3 bucket naam: datasource-klantnaam
AWS access key id
AWS secret access key
AWS region: eu-central-1 (Frankfurt)
5. PowerBI desktop Python connector (S3 bucket)
5. PowerBI desktop Python connector (S3 bucket)
Operating System: Linux, Windows, and macOS
Tested on macOS 15.2 with Cyberduck Client 9.1.0
Requirements:
AWS CLI
AWS S3 bucket name: datasource-[customer_name]
AWS access key ID
AWS secret access key
AWS region: eu-central-1 (Frankfurt)
Open command prompt (cmd.exe)
# ga naar de folder waar python is geïnstalleerd
cd C:\Users\xxxxxx\AppData\Local\Programs\Python\Python311
# install python package manager pip
curl https://bootstrap.pypa.io/ez_setup.py | python
curl https://bootstrap.pypa.io/get-pip.py | python
# install required python modules
Scripts\pip.exe install boto3 matplotlib pandas
Open PowerBI Desktop > Home > Get data > more > Python
Crcompany.csv example
import boto3, os, io
import pandas as pd
my_key= 'xxxxxx'
my_secret= 'xxxxxx'
my_bucket_name = 'datasource-xxxxxx'
my_file = 'Crcompany.csv'
session = boto3.Session(aws_access_key_id=my_key,aws_secret_access_key=my_secret)
s3 = session.resource('s3')
bucket = s3.Bucket(my_bucket_name)
bucket.download_file(my_file, my_file)
crcompany = pd.read_csv(my_file, delimiter='\t')
6. AWS command line interface (S3 bucket)
6. AWS command line interface (S3 bucket)
Operating System: Linux, Windows, and macOS
Tested on macOS 15.2 with AWS CLI 2.22.17
Requirements:
AWS CLI
AWS S3 bucket name: datasource-[customer_name]
AWS access key ID
AWS secret access key
AWS region: eu-central-1 (Frankfurt)
# macOS example
$ which aws
/usr/local/bin/aws
$ aws --version
aws-cli/2.2.23 Python/3.8.8 Darwin/22.6.0 exe/x86_64 prompt/of
# create default profile
$ aws configure
AWS Access Key ID [None]: xxxxxx
AWS Secret Access Key [None]: xxxxxx
Default region name [None]: eu-central-1
Default output format [None]: json
$ aws s3 ls s3://datasource-klantnaam/
2023-08-19 03:32:06 8456 Cragency.csv
2023-08-19 03:32:06 4579 Crarticle.csv
2023-08-19 03:32:06 153351138 Crattachment.csv
2023-08-19 03:32:08 509534996 Crattributechange.csv
...
# sync all csv files to local folder /var/tmp
# Only download files if they have been modified; this prevents unnecessary downloads.
$ aws s3 sync s3://datasource-klantnaam/ /var/tmp --include "*.csv"