Questions tagged [aws]
AWS refers to Amazon Web Services.
169 questions
2
votes
2
answers
132
views
How to deal with a poorly documented AWS SQS feed for a security product API?
I’m integrating with the Ajax Enterprise API which exposes an AWS SQS feed for device events and notifications. The API documentation is here: Ajax Enterprise API Docs.
Each SQS message supplies an ...
- 129
0
votes
0
answers
93
views
How to connect to SFTP using Apache Spark 3.5 with Scala 2.12 for parallel file transfers?
I am working on a project where I need to transfer thousands of files (each sized between 50-60 MB) every hour from an SFTP server to local storage or AWS S3. I am using Apache Spark 3.5 with Scala 2....
- 11
0
votes
1
answer
132
views
How to model a bidirectional, one-to-one relationship in DynamoDB?
I'm building a WebSocket API Gateway in AWS. When a device connects, API Gateway assigns the connection a unique connection ID. I need to store the relationship between the connection ID and device ...
- 1
1
vote
2
answers
528
views
Is it worth it to use a NoSQL database only for high availability and fast reading, while already using Postgres?
We have several Postgres databases on one AWS RDS cluster (for budget reasons apparently), sharing the same resources, and my team wants to use a NoSQL database alongside it, and store data there from ...
- 113
1
vote
1
answer
160
views
Designing a Kafka consumer that gets a high throughput of "redundant" events
Context:
We have 2 services - a user-facing API server and a background processing Kafka worker.
Both of these are hosted on AWS EKS, and we use AWS MSK for Kafka.
The job of the API server is to ...
- 111
0
votes
0
answers
72
views
Most efficient way to maintain versioned business rules files for an Amazon ECS service?
I have an Amazon ECS service that relies on configuration files that contain business rules. The business rules are changed periodically by non-programmers. Total size of the rules files is ...
- 131
0
votes
1
answer
466
views
Is it a good idea to serve a Single Page Application as a static site?
I've built a small single-page web application in React and seen that it's possible to serve the app as a static site on something like S3.
Previously, I considered using Nginx, but as this is lower ...
- 159
0
votes
0
answers
67
views
Looking for patterns for instantaneous data consistency for DB transactions that span multiple schemas in a Aurora postgres instance
We are currently developing a product in the AWS environment that requires instantaneous data consistency for DB transactions that span multiple schemas in a Aurora postgres instance. Each schema will ...
1
vote
2
answers
347
views
Using Fargate (serverless compute) to replace leadership election
I currently have a solution that requires leadership election for HA. It processes files as they arrive in an SFTP server, and Subscribe to an Active MQ feed; and therefore we don't want multiple ...
1
vote
2
answers
186
views
Creating associated resources before creating the main one in a REST setting
I'm not sure how common of a scenario this is, but what are some approaches to creating associated entities before you have the actual main entity they're supposed to be linked to? (basically having ...
-2
votes
2
answers
241
views
Quiz Site Software Architecture Question
I am currently writing an application that is going to be an adaptive quiz-like site for studying. The idea is a user is studying some topic and they are given questions on the site and enter their ...
- 11
0
votes
1
answer
188
views
AWS and cron job: improve logging, monitoring and concurrence
In my company, we use a separate cron server to run mission-critical background jobs that run on a single ec2 instance: the whole platform is then vulnerable to anything going wrong on this instance.
...
-1
votes
2
answers
155
views
Architecture for EOD (end of day) stock exchange prices
I need to work out the architecture for a NASDAQ frontend charting application (a desktop app in .Net). Note that this is NOT for real-time quotes.
NASDAQ provides an api that gives historical pricing,...
- 117
0
votes
0
answers
962
views
Transactionally update a set of files in s3
I have a bucket with several csv files at differing paths. The problem is that I want to update these csv files transactionally. The s3 bucket should represent the "latest" consistent set of ...
- 345
1
vote
0
answers
875
views
AWS multiple SQS queues on one bucket without fanout
I am coming into a project that uses an Amazon S3 Bucket for a process.
For my application, I need to know (within a reasonable time - order of a few minutes, at most) whether a new file has beeen ...
- 161