Skip to content

Andrew Smith

Software Engineer / Architect

  • Home
  • Bio/Contact

Month: July 2016

Leveraging the Database Engine

Leveraging the Database Engine

In some situations it’s appropriate to push operations down to the underlying RDBMS, that could have otherwise been performed by SSIS.

This is subject to the RDBMS having the capacity available to perform these operations, but assuming it does, then this is a useful option to have to hand […] “Leveraging the Database Engine”

Author Andrew SmithPosted on Posted on 29th July 201626th May 2020Categories UncategorisedTags ETL, SQL Server, SSIS
SSIS Unit Testing

SSIS Unit Testing

Testing of SSIS packages can be an ah-hoc, manual process that’s performed only at the package level. I.e. checking that running a whole package (or set of packages) produces the expected data.

This approach will likely not test all paths through an ETL, as well as failing to thoroughly test priority fixes […] “SSIS Unit Testing”

Author Andrew SmithPosted on Posted on 22nd July 201626th May 2020Categories UncategorisedTags SQL Server, SSIS
Data Flow Threading

Data Flow Threading

SSIS provides for parallelism at multiple levels: (i) by running packages concurrently, (ii) by running tasks concurrently within a package, and (iii) by running transformations concurrently within a data flow.

This post investigates parallelism within the data flow, by comparing the time taken between non-parallel and […] “Data Flow Threading”

Author Andrew SmithPosted on Posted on 14th July 201626th May 2020Categories UncategorisedTags ETL, SQL Server, SSIS

Posts navigation

Page 1 Page 2 Older Posts

About

I’m Andrew Smith and I’m an experienced software professional with an employment history spanning a variety of industries and including blue-chip organisations such as Ford Motor Company, PricewaterhouseCoopers and IQVIA.

I’m currently architecting and implementing an AWS-based IoT system, including network and security design, RDS database implementation and the provision of high availability and disaster recovery capabilities.

Whilst this website was initially written to document research I was doing throughout the Microsoft SQL Server product suite, it’s now oriented around how best to use Amazon Web Services (AWS) to deliver solutions to problems in enterprise computing.

Contact

Email:
andrew@andrewleesmith.co.uk
Linked In:

Popular Tags

  • AWS (8)
  • ETL (20)
  • Python (6)
  • SQL Server (38)
  • SSIS (17)

Recent Posts

  • Optimising Query Execution for Amazon Redshift Spectrum
  • Processing IoT Data using Amazon Kinesis
  • Running ETL Jobs using Amazon Athena
  • Integrating AWS Glue Jobs with S3 Notifications and EventBridge
  • Loading data into Amazon Redshift using AWS Glue

Archives

  • June 2020
  • May 2020
  • April 2020
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • August 2016
  • July 2016
  • Home
  • Bio/Contact
Andrew Smith Proudly powered by WordPress
Close Bitnami banner
Bitnami