Our customers made these suggestions for improving Etlworks Integrator. Suggestions for improving Etlworks Integrator.
44
votes
Packages and versioning
It would be nice to create a "package" of flows and connections and store it as a certain version. It should then be easy to switch between versions just by toggling ...
32
votes
2FA Login
Our security policy will require all tools that have access to company data, to have 2FA required to login.
Currently a user can login using only the username and ...
Done
10
votes
SSO Authentication
Etlworks handles and processes a lot of sensitive data. It is important that this is adequately shielded and becomes part of our company-wide security umbrella. The ...
9
votes
Drag and drop process builder for nested flows
Replace existing nested flow editor with drag and drop process builder:
- works similarly to the diagram editor
- ability to select existing flows from the side-bar ...
8
votes
Usability enhancement for Flows Builder
1. Ability to edit inner flow from the nested flow
2. Ability to stop API flow, also stop flow from scheduler
3. Add Find Usage for flows (within nested flows)
8
votes
New fields' mapping editor
Enhance field mapping editor:
- add support for nested fields in the destination
- add an ability to modify attributes, such as JSON array, XML CDATA, XML ...
7
votes
Bulk edit configuration mapping
Instead of updating source/destination field names or data types individually, allow selection of desired fields for bulk modification(s).
Under consideration
7
votes
Provisioning from AWS and Azure Marketplaces
Add ability to provision Etlworks from AWS and Azure Marketplaces with Amazon and Microsoft billing
6
votes
Email on schedule failure only if last retry
I don't want emails for transient errors if a retry worked
Done
6
votes
IBM i Debezium Connector
Hello, would it be possible to add IBM i/AS400 Debezium Connector capabilities via this repo please?
https://github.com/jhc-systems/debezium-connector-ibmi
6
votes
Ability to schedule and execute flows by data integration agents running behind the firewall
Basically, add support for hybrid data integration when flows can be executed on data integration agents running behind the firewall.
6
votes
Test Script button
Testing the „Execute JavaScript or Python code“ type flows is very troublesome, as you have to save (which closes the modal), press run, open the flow execution view, ...
Under consideration
6
votes
(re-)Run from error screen
If investigate an issue within ETLworks. You go to the error (via status) to the statistics section. When solving the issue, you have to search for the flow in order ...
Planned
5
votes
Better ways to determine how flows, connections, etc. are interconnected
Right now when you have multiple flows that share connections or other flows, it's difficult to understand how making a change to one will possibly impact other ...
5
votes
Implement priority queue and throttling for user-defined APIs (listeners)
Scenario 1: upstream system slammed the system with thousands of API calls in a short span of time, eventually freezing and requiring a reboot to stop all jobs ...
5
votes
4
votes
Connector for Google Analytics 4,
Since Google is discontinuing Universal Analytics and moving to only supporting Google Analytics 4,
4
votes
AWS Kinesis Integration
AWS Kinesis Integration
4
votes
Search functionality / better documentation of flows
We use a good deal of javascript and in complex flows it can be really difficult to debug, esp. global and flow variables because there's no way to search (that I ...
4
votes
Add exceptions/exclusions rules to schedules
Often times we need to apply schedules to not run at certain times for various reasons. Right now, we have to create multiple schedules. It would be nice to ...
4
votes
View of schedules across all tenants + Last execution duraton
A view were all schedules across all tenants are visible with additional column with last execution duration will help balance load across instance. It will help pick ...
4
votes
Auto-completion in JavaScript code editors
Add auto-completion in JavaScript code editors:
- {field names}
- functions
3
votes
Scheduler queue
Introduce queuing in scheduler so you can set how many scheduled tasks can execute simultaneously, and if one tries to run while another is running it will wait till ...
3
votes
Allow creation of new connections for each parallel execution in Bulk to Synapse type flows.
Currently, in Nested flows, when executing consecutive SQL Script flows with "Parallel" nested flow option turned on, new connection is created for each SQL Script ...
Done
3
votes
Bulk insert with db to db flows
Having inserts into postgres and sqlserver work like redshift would be a nice improvement. Right now it runs a bunch of insert statements. Instead, pull source data ...
3
votes
Add ability to resume a sync from where it might have failed, or was stopped/canceled
Sometimes syncs fail for whatever reasons, but being able to easily resume from the flows that were not completed would be fantastic (assuming we separately addressed ...
Done
3
votes
Import entire schemes without specifying individual table
I really liked how aws dms operated, you pointed it at a schema on your source, told it the target schema in your destination, and had a wildcard supporting ...
3
votes
UI Suggestion: Flow execution/failure duration metrics
Currently, some flows only have a console log of events that happen during flow execution. It would be useful to have a graphical representation, especially nested ...
Done
3
votes
Table view in Test Transformation
1) If the destination connection is a database display output in a table view
2) Use code editor for a preview
3
votes
Ability to create and share templates
Templates are similar to export/import. Additionally, users should be able to:
- Create a template from any Flow/Connection/Format.
- Share templates with everyone ...
3
votes
Repeat long running schedule notifications
When telling a schedule to alert you if it’s been running for more than X minutes, allow it to alert you every X minutes. Right now I can’t tell the difference ...
Under consideration
3
votes
FLOWs visual based on tree hierarchy
When having multiple flows, it would be nice if you can collapse actions into the nested flows (like a tree).
Result then, only nested flows will be shown, until ...
Under consideration
3
votes
Add interactive Azure Active directory authentication for SQL Server connector
SQL Server JDBC driver supports accessToken authentication which requires a web flow for signing in with Microsoft. It would be great if Etlworks connector could ...
Done
3
votes
Support IAM For S3 Authentication
We run ETL Works on a self-hosted EC2 server. To be able to connect to AWS S3 from ETL Works, it currently requires an access key and secret key. Our organisation ...
Done
3
votes
Allow setting flow parameter in explorer
Made a api connector with a {param1} in the url, want to do this in explorer: select * from content where param1=123
3
votes
Add a next/previous transform button when editing transform details
Easily transition between transforms to more quickly apply a change to all transforms without losing your place in the list going back and fourth
2
votes
Statistics Page Should include Agent Metrics too
Currently Statistics page does not include metrics for the flows which are executed by Agent... it would make dashboard much better if we can see the whole picture ...
2
votes
Add run schedule api endpoint
I'd like to run schedule job via api instead of just flow. This way I can take advantage of the scheduler's extra options like expected flow execution time. Pass ...
Done
2
votes
Recreating table for HWM flow after data is populated
On a non-HWM transform with new source columns, the destination table isn’t dropped until after the data is populated. On a HWM transform it drops it before data ...
Under consideration
2
votes
Allow for table list alphabetization in explorer
Expanded salesforce tables in explorer, table names are not alphabetized, makes it hard to find anything
2
votes
Searching flows should also search table names
I want to find a flow that syncs a specific table, the search only includes the flow names. If it's easy include the table names or other sub-fields or scripts
2
votes
Monospace default font in editors
Set default font in all editors to monospace, or allow customization option somewhere (either at bottom of editor or at profile level, preferably both).
Done
2
votes
dashboard of tenants overview and alerts
We would like to have a dashboard with statistics on all the tenants on one screen for managing and keeping the resources allocated inline
1. how many failures have ...
Done
2
votes
Add button to restart service
Sometimes there is a need to restart ETL works service, it would be nice to have a button or something like that in the interface
Not planned
2
votes
Add ability to generate new Google Sheets
I'm looking for a way to generate new Google Sheets from ETLworks.
Or at minimum, use the Google Drive connector to create a new file, then the Google Sheets ...
2
votes
Parallel CDC snapshot
Currently, CDC flows run snapshots sequentially, which for large databases can be a bottleneck. It would be great to have an option to run snapshots in parallel, each ...
Done
2
votes
support parameterized macros
allow passing params directly to macros when referencing them
{{4:MacroName}('MySpecialParam')}
or something like that
Not planned