Our customers made these suggestions for improving Etlworks Integrator. Suggestions for improving Etlworks Integrator.
44
votes
Packages and versioning
It would be nice to create a "package" of flows and connections and store it as a certain version. It should then be easy to switch between versions just by toggling ...
10
votes
SSO Authentication
Etlworks handles and processes a lot of sensitive data. It is important that this is adequately shielded and becomes part of our company-wide security umbrella. The ...
9
votes
Drag and drop process builder for nested flows
Replace existing nested flow editor with drag and drop process builder:
- works similarly to the diagram editor
- ability to select existing flows from the side-bar ...
8
votes
Usability enhancement for Flows Builder
1. Ability to edit inner flow from the nested flow
2. Ability to stop API flow, also stop flow from scheduler
3. Add Find Usage for flows (within nested flows)
8
votes
New fields' mapping editor
Enhance field mapping editor:
- add support for nested fields in the destination
- add an ability to modify attributes, such as JSON array, XML CDATA, XML ...
7
votes
Provisioning from AWS and Azure Marketplaces
Add ability to provision Etlworks from AWS and Azure Marketplaces with Amazon and Microsoft billing
6
votes
Better ways to determine how flows, connections, etc. are interconnected
Right now when you have multiple flows that share connections or other flows, it's difficult to understand how making a change to one will possibly impact other ...
6
votes
IBM i Debezium Connector
Hello, would it be possible to add IBM i/AS400 Debezium Connector capabilities via this repo please?
https://github.com/jhc-systems/debezium-connector-ibmi
6
votes
Ability to schedule and execute flows by data integration agents running behind the firewall
Basically, add support for hybrid data integration when flows can be executed on data integration agents running behind the firewall.
5
votes
Implement priority queue and throttling for user-defined APIs (listeners)
Scenario 1: upstream system slammed the system with thousands of API calls in a short span of time, eventually freezing and requiring a reboot to stop all jobs ...
5
votes
Add exceptions/exclusions rules to schedules
Often times we need to apply schedules to not run at certain times for various reasons. Right now, we have to create multiple schedules. It would be nice to ...
5
votes
Search functionality / better documentation of flows
We use a good deal of javascript and in complex flows it can be really difficult to debug, esp. global and flow variables because there's no way to search (that I ...
5
votes
5
votes
Auto-completion in JavaScript code editors
Add auto-completion in JavaScript code editors:
- {field names}
- functions
4
votes
Connector for Google Analytics 4,
Since Google is discontinuing Universal Analytics and moving to only supporting Google Analytics 4,
4
votes
AWS Kinesis Integration
AWS Kinesis Integration
4
votes
View of schedules across all tenants + Last execution duraton
A view were all schedules across all tenants are visible with additional column with last execution duration will help balance load across instance. It will help pick ...
3
votes
Scheduler queue
Introduce queuing in scheduler so you can set how many scheduled tasks can execute simultaneously, and if one tries to run while another is running it will wait till ...
3
votes
Bulk insert with db to db flows
Having inserts into postgres and sqlserver work like redshift would be a nice improvement. Right now it runs a bunch of insert statements. Instead, pull source data ...
3
votes
Add message and script to scheduler email
I want to add a message to the scheduler that contains all the handled flow exceptions.
Add a message field where we can append additional text to the default email ...
3
votes
Import entire schemes without specifying individual table
I really liked how aws dms operated, you pointed it at a schema on your source, told it the target schema in your destination, and had a wildcard supporting ...
3
votes
Table view in Test Transformation
1) If the destination connection is a database display output in a table view
2) Use code editor for a preview
3
votes
Ability to create and share templates
Templates are similar to export/import. Additionally, users should be able to:
- Create a template from any Flow/Connection/Format.
- Share templates with everyone ...
3
votes
Allow setting flow parameter in explorer
Made a api connector with a {param1} in the url, want to do this in explorer: select * from content where param1=123
3
votes
Add a next/previous transform button when editing transform details
Easily transition between transforms to more quickly apply a change to all transforms without losing your place in the list going back and fourth
2
votes
Statistics Page Should include Agent Metrics too
Currently Statistics page does not include metrics for the flows which are executed by Agent... it would make dashboard much better if we can see the whole picture ...
2
votes
Allow for table list alphabetization in explorer
Expanded salesforce tables in explorer, table names are not alphabetized, makes it hard to find anything
2
votes
Searching flows should also search table names
I want to find a flow that syncs a specific table, the search only includes the flow names. If it's easy include the table names or other sub-fields or scripts
2
votes
Add runtime per step (addition to start / stop)
A this moment we have to interpretate the flowstep running time by calculating Start / Stop . It should be handy if you could see this at once. We use this qiut often.
2
votes
Interactive Performance recommandations
It would be a good idea to get performance recommandations based on one or more (nested) flows. Goal is decrease (processing time) of the flows. Maybe this can be ...
2
votes
Add ability to generate new Google Sheets
I'm looking for a way to generate new Google Sheets from ETLworks.
Or at minimum, use the Google Drive connector to create a new file, then the Google Sheets ...
2
votes
Implement Kafka Headers
Please consider implementing Kafka Headers. Thanks!
2
votes
Add ability to filter flow executions by status
Is it possible to filter events in the flow dashboard by status: success, error, etc?
2
votes
Execute as script in nested SQL loop
I need to execute a proc from a nested loop script but I get “executeQuery only accepts the SELECT statement. Use executeUpdate instead.”
Maybe add an ‘execute as ...
2
votes
Connector for Google User Activity API
I want to get some data from the google api: https://analyticsreporting.googleapis.com/v4/userActivity:search via oath2.0
I manage to get this done via postman.
2
votes
Create subfolders when saving CDC events in cloud storage
Can you add the ability to automatically create sub-folders in S3 when saving CDC events?
2
votes
Insert all CDC events into a single topic in the message queue
We are streaming CDC events into the Azure Messages Hubs. Currently, Etlworks creates one topic per database.table. Is it possible to insert all events into a single ...
2
votes
Retain Debezium 'before' element
We are steaming CDC events into the Azure Events Hubs. The events are getting 'flatten' by Etlworks. Is it possible to retain the original structure of the CDC event ...
2
votes
Call Greenplum gpload from Etlworks
We are currently using Greenplum gpload utility to load data into on-premise Greenplum database. Is it possible to integrate it with Etlworks?
2
votes
Improve loading data into Google BigQuery
We are currently using database-to-database flow to load data into Google Query. It works but it is slow. It would be great if you could add BogQuery optimized flows, ...
2
votes
Run as script from explorer
Allow run as script from explorer, for multi-line scripts
2
votes
Running tasks copy request
How about a copy button for each task to copy the script to clipboard? I’ve had to do that a few times now
2
votes
Add content encoding to gzip files copied to s3
postgres aws_s3.table_import_from_s3 requires content-encoding to be set on gzip files stored in s3. I had to manually add content-encoding header to the s3 ...
2
votes
Use datetime2 for sql server instead of datetime
Somehow someone in my postgres database was born in the year 1050. Old mofo
2
votes
Max runtime value for scheduled job
Add a field if scheduled job runs over X minutes automatically kill it. Had a job hang 17 hours, some sort of lock on internal logging I think.
2
votes
UI/UX enhancements
1. By default filter out users from deactivated tenants
2. Add right mouse click in Explorer
3. Do not display "scheduled" icon (or display it as grayed out) for ...