Suggestions for improving Etlworks Integrator.

44 votes

Packages and versioning

It would be nice to create a "package" of flows and connections and store it as a certain version. It should then be easy to switch between versions just by toggling ...
Suggested by: Franck van der Sluis (19 Apr, '19) Upvoted: 28 Jan Comments: 3
Done roadmap
18 votes

The ability to manage tags: edit, delete

Suggested by: Rupesh Sawant (23 Jun, '20) Upvoted: 04 Feb Comments: 1
Planned roadmap
11 votes

GUI Database Lookup

As a user, creating database lookups requires the user to know code in the field value function to perform database lookups. This is even more complicated when doing ...
Suggested by: Ryan Cronin (10 Apr, '24) Upvoted: 29 Oct, '24 Comments: 2
Planned roadmap
10 votes

SSO Authentication

Etlworks handles and processes a lot of sensitive data. It is important that this is adequately shielded and becomes part of our company-wide security umbrella. The ...
Suggested by: Joris Hooi (24 Oct, '19) Upvoted: 05 May, '22 Comments: 1
Done roadmap
9 votes

Test Script button

Testing the „Execute JavaScript or Python code“ type flows is very troublesome, as you have to save (which closes the modal), press run, open the flow execution view, ...
Suggested by: Heiko (10 Aug, '23) Upvoted: 12 Jul, '24 Comments: 0
Planned roadmap
8 votes

(re-)Run from error screen

If investigate an issue within ETLworks. You go to the error (via status) to the statistics section. When solving the issue, you have to search for the flow in order ...
Suggested by: Marvin (27 Feb, '23) Upvoted: 23 Jan Comments: 0
Planned roadmap
8 votes

New fields' mapping editor

Enhance field mapping editor: - add support for nested fields in the destination - add an ability to modify attributes, such as JSON array, XML CDATA, XML ...
Suggested by: Olena Sherbinin (06 Dec, '18) Upvoted: 26 Jan, '24 Comments: 0
Done roadmap
7 votes

Add retry functionality to agent schedules

ETLWorks host based schedules support: retries (1 - 5), delay between retries (1 - 10 minutes) and send failure email only if last retry failed. It would be really ...
Suggested by: José (09 Nov, '24) Upvoted: 25 Nov, '24 Comments: 1
Done roadmap
7 votes

Provisioning from AWS and Azure Marketplaces

Add ability to provision Etlworks from AWS and Azure Marketplaces with Amazon and Microsoft billing
Suggested by: Olena Sherbinin (06 Dec, '18) Upvoted: 16 Mar, '24 Comments: 0
Done roadmap
6 votes

Better ways to determine how flows, connections, etc. are interconnected

Right now when you have multiple flows that share connections or other flows, it's difficult to understand how making a change to one will possibly impact other ...
Suggested by: Shanon (07 Jul, '19) Upvoted: 29 Oct, '24 Comments: 5
Done roadmap
6 votes

IBM i Debezium Connector

Hello, would it be possible to add IBM i/AS400 Debezium Connector capabilities via this repo please? https://github.com/jhc-systems/debezium-connector-ibmi
Suggested by: Ryan Cronin (01 Feb, '23) Upvoted: 16 Jan, '24 Comments: 1
Done roadmap
6 votes

Ability to schedule and execute flows by data integration agents running behind the firewall

Basically, add support for hybrid data integration when flows can be executed on data integration agents running behind the firewall.
Suggested by: Olena Sherbinin (23 Mar, '19) Upvoted: 27 Dec, '21 Comments: 1
Done roadmap
6 votes

Ability to update all or selected agents to the specific version

Currently each agents needs to be updated individually. From our perspective, for bulk updating, I would say it would be managed via the Agents page itself. - ...
Suggested by: Olena Sherbinin (26 Jan) Upvoted: 2 days ago Comments: 0
Planned roadmap
6 votes

Improved Filter/Search in Flow Statistics Page

It can be cumbersome to locate a specific job in the Flow Statistics page since it loads the jobs in the order in which it was run. Being able to search or filter ...
Suggested by: Jason Saltzman (25 Oct, '24) Upvoted: 23 Jan Comments: 0
Planned roadmap
6 votes

Recycle Bin when deleting flows (or other assets)

It would be nice if there was a Recycle Bin for flows (or other assets), and it could have a time limit (like 30 days). This way, if one of our users accidentally ...
Suggested by: Brad (06 Aug, '24) Upvoted: 28 Jan Comments: 0
Planned roadmap
6 votes

Drag&drop SQL query builder and Field Function Bilder

- SQL Builder (in Flow and Explorer) - Field Function Builder (at least Lookup)
Suggested by: Olena Sherbinin (13 Mar, '19) Upvoted: 23 Jan Comments: 0
Planned roadmap
5 votes

Implement priority queue and throttling for user-defined APIs (listeners)

Scenario 1: upstream system slammed the system with thousands of API calls in a short span of time, eventually freezing and requiring a reboot to stop all jobs ...
Suggested by: Olena Sherbinin (07 Nov, '20) Upvoted: 08 Mar, '21 Comments: 3
Done roadmap
5 votes

Add exceptions/exclusions rules to schedules

Often times we need to apply schedules to not run at certain times for various reasons. Right now, we have to create multiple schedules. It would be nice to ...
Suggested by: Ryan Cronin (08 Jul, '23) Upvoted: 25 Apr, '24 Comments: 1
Done roadmap
5 votes

Search functionality / better documentation of flows

We use a good deal of javascript and in complex flows it can be really difficult to debug, esp. global and flow variables because there's no way to search (that I ...
Suggested by: Shanon (31 Oct, '19) Upvoted: 29 Oct, '24 Comments: 1
Done roadmap
5 votes

Truly dynamic connection and format

Currently adding dynamic connection and format requires actually adding a named connection to the flow. It would be great if both could be truly dynamic, resolved at ...
Suggested by: Olena Sherbinin (28 Jan) Upvoted: 28 Jan Comments: 0
Planned roadmap
5 votes

Append Mode Option in Flows

Feature Name: Append Mode Purpose: To provide a seamless way to append data to the destination without overwriting existing records. Implementation Suggestion: A ...
Suggested by: Suraj Deval (16 Jan) Upvoted: 21 Jan Comments: 0
Planned roadmap
5 votes

Statistics Executions: Sort and Filter Updates

On the statistics page, specifically the Executions area. The ability to: 1. Sort the results by the column headers. At least the started, ended, duration, and ...
Suggested by: Jeff Merino-Ott (07 Oct, '24) Upvoted: 28 Jan Comments: 0
Planned roadmap
4 votes

Ability to purge files within Server Storage connection after specified number of days

I would like to be able to delete files within a Server Storage connection after a specific number of days. This can be useful if I generate file logs into a specific ...
Suggested by: Brad (30 Aug, '24) Upvoted: 17 Jan Comments: 3
Done roadmap
4 votes

Interactive Performance recommandations

It would be a good idea to get performance recommandations based on one or more (nested) flows. Goal is decrease (processing time) of the flows. Maybe this can be ...
Suggested by: Michel van Drunen (19 Jun, '24) Upvoted: 23 Jan Comments: 1
Planned roadmap
4 votes

Connector for Google Analytics 4,

Since Google is discontinuing Universal Analytics and moving to only supporting Google Analytics 4,
Suggested by: Olena Sherbinin (18 Feb, '23) Upvoted: 18 Feb, '23 Comments: 1
Done roadmap
4 votes

AWS Kinesis Integration

AWS Kinesis Integration
Suggested by: Karthik (08 Jun, '20) Upvoted: 14 Jul, '20 Comments: 1
Done roadmap
4 votes

Add message and script to scheduler email

I want to add a message to the scheduler that contains all the handled flow exceptions. Add a message field where we can append additional text to the default email ...
Suggested by: Bryan (24 Mar, '20) Upvoted: 23 Jan Comments: 1
Planned roadmap
4 votes

New mapping options

1. If configured throw an error if there is column in the source which does not exist in mapping 2. If configured ignore all columns in the source which are not in ...
Suggested by: Olena Sherbinin (28 Jan) Upvoted: 28 Jan Comments: 0
Planned roadmap
4 votes

Filtering audit logs by time

Currently, audit logs can be filtered by the usable fields and a date range. Having a large amount of entry's, makes the log hard to use with these specifications, if ...
Suggested by: Heiko Parmas (06 Dec, '24) Upvoted: 2 days ago Comments: 0
Planned roadmap
4 votes

Excluding schedules or flows from the execution notification

I do like having the execution notification in the top right corner of the page, however there are some scheduled tasks which run so frequently that it would make the ...
Suggested by: Brad (06 Aug, '24) Upvoted: 23 Jan Comments: 0
Planned roadmap
4 votes

View of schedules across all tenants + Last execution duraton

A view were all schedules across all tenants are visible with additional column with last execution duration will help balance load across instance. It will help pick ...
Suggested by: Jaroslaw (20 Nov, '20) Upvoted: 28 Jan, '24 Comments: 0
Done roadmap
3 votes

Scheduler queue

Introduce queuing in scheduler so you can set how many scheduled tasks can execute simultaneously, and if one tries to run while another is running it will wait till ...
Suggested by: Bryan (18 Mar, '20) Upvoted: 12 Oct, '22 Comments: 2
Done roadmap
3 votes

Add support for JMS, ActiveMQ and AWS SQS

Suggested by: Maksym Sherbinin (03 Oct, '19) Upvoted: 14 Jul, '21 Comments: 2
Done roadmap
3 votes

Bulk insert with db to db flows

Having inserts into postgres and sqlserver work like redshift would be a nice improvement. Right now it runs a bunch of insert statements. Instead, pull source data ...
Suggested by: Bryan (10 Oct, '20) Upvoted: 15 Nov, '20 Comments: 1
Done roadmap
3 votes

Import entire schemes without specifying individual table

I really liked how aws dms operated, you pointed it at a schema on your source, told it the target schema in your destination, and had a wildcard supporting ...
Suggested by: Bryan (24 Jul, '19) Upvoted: 28 Sep, '19 Comments: 1
Done roadmap
3 votes

Table view in Test Transformation

1) If the destination connection is a database display output in a table view 2) Use code editor for a preview
Suggested by: Maksym Sherbinin (10 Jan, '19) Upvoted: 03 Nov, '23 Comments: 1
Done roadmap
3 votes

Ability to create and share templates

Templates are similar to export/import. Additionally, users should be able to: - Create a template from any Flow/Connection/Format. - Share templates with everyone ...
Suggested by: Olena Sherbinin (06 Dec, '18) Upvoted: 11 Dec, '23 Comments: 1
Done roadmap
3 votes

Multi select flows for bulk actions (delete, add tags, move to tenant) etc.

Would like to be able to select via checkbox or "all" results of a search and perform a bulk action on a set of flows, connections, users, etc.
Suggested by: SG (21 Oct, '24) Upvoted: 23 Jan Comments: 0
Planned roadmap
3 votes

Add compression to copy files flow

Suggested by: Bryan (12 Nov, '23) Upvoted: 23 Jan Comments: 0
Planned roadmap
3 votes

Default tenant at login

Allow user to set default tenant (e.g., dev versus prod) when logging in. Currently defaults to no tenant.
Suggested by: Jonathon (03 Nov, '23) Upvoted: 23 Jan Comments: 0
Planned roadmap
3 votes

Allow for export of Record Metrics from statistics page

Suggested by: Bryan (19 May, '21) Upvoted: 23 Jan Comments: 0
Planned roadmap
3 votes

Allow setting flow variables in scheduler

Suggested by: Bryan (27 Mar, '20) Upvoted: 05 Oct, '20 Comments: 0
Done roadmap
2 votes

Statistics Page Should include Agent Metrics too

Currently Statistics page does not include metrics for the flows which are executed by Agent... it would make dashboard much better if we can see the whole picture ...
Suggested by: Atul Kapoor (09 May, '22) Upvoted: 09 May, '22 Comments: 3
Done roadmap
2 votes

Allow for table list alphabetization in explorer

Expanded salesforce tables in explorer, table names are not alphabetized, makes it hard to find anything
Suggested by: Bryan Meyerovich (10 Jun, '20) Upvoted: 10 Jun, '20 Comments: 2
Done roadmap
2 votes

Searching flows should also search table names

I want to find a flow that syncs a specific table, the search only includes the flow names. If it's easy include the table names or other sub-fields or scripts
Suggested by: Bryan (12 Feb, '20) Upvoted: 18 Mar, '20 Comments: 2
Done roadmap
2 votes

Add runtime per step (addition to start / stop)

A this moment we have to interpretate the flowstep running time by calculating Start / Stop . It should be handy if you could see this at once. We use this qiut often.
Suggested by: Marvin (11 Jul, '24) Upvoted: 11 Jul, '24 Comments: 1
Done roadmap
2 votes

Add ability to generate new Google Sheets

I'm looking for a way to generate new Google Sheets from ETLworks. Or at minimum, use the Google Drive connector to create a new file, then the Google Sheets ...
Suggested by: Olena Sherbinin (30 Nov, '21) Upvoted: 30 Nov, '21 Comments: 1
Done roadmap
2 votes

Feature request: smb storage connector through ssh bastion

Suggested by: Bryan (27 Apr, '21) Upvoted: 27 Apr, '21 Comments: 1
Done roadmap
2 votes

Can register with same email on multiple tenants

As administrator, I would like to use the same email on multiple tenants as opposed to having to create a new one per tenant. This would also require that email ...
Suggested by: Bharath (22 Oct, '19) Upvoted: 20 Nov, '24 Comments: 1
Planned roadmap
2 votes

Implement Kafka Headers

Please consider implementing Kafka Headers. Thanks!
Suggested by: Ryan Cronin (20 Jul, '22) Upvoted: 20 Jul, '22 Comments: 0
Done roadmap