Showing posts with label SSIS. Show all posts
Showing posts with label SSIS. Show all posts

Wednesday, September 4, 2013

SSIS passing DATE datatype into Exe SQL Task via Store Procedure

This is a really simple question:

There is one store procedure created on production COS US, now I want to call it via SSIS.

DECLARE @retailerID INT
DECLARE @date      DATE
DECLARE @column_empty  BIT
DECLARE @noData       BIT
DECLARE @cnt        INT
DECLARE @uploads    INT 
DECLARE @exists     BIT
DECLARE @sp_status     INT

SET @retailerid = 27
SET @date  = CONVERT(DATE,'20130903')

  
EXEC @sp_status = dbo.PopulateCostoUS_TransactionFee  @retailerID  ,  @date , @column_empty  OUTPUT, @cnt  OUTPUT, @uploads  OUTPUT,@noData     OUTPUT

SELECT    @retailerID as retailer    ,DATEADD(day,-1,@date)  as [date], @column_empty as column_empty, @cnt as cnt, @uploads as uploads ,@noData as NoDataFlag

 

HOW CAN I DO IT? It’s causing a lot of pains now…????

 

This BLOG saved my life

 

http://blog.hoegaerden.be/2011/06/11/passing-a-datetime-parameter-into-the-execute-sql-task/

 

 

Basically, if the package variable is using DATETIME type , then, in the Parameter mapping section, the data type should be DATE which  consists of year, month, day, hour, minute, seconds, and fractional seconds. The fractional seconds have a fixed scale of 7 digits.

if the SP is using INT and the pacakge variables that’s going to be passed to this parameter is INT32 , then parameter mapping should be LONG, instead of short

In expression, if you want to show the value of the variable, use double quote “, if you want to put the value under the single quote , use +’” @val +”’

Tuesday, July 16, 2013

How to parse XML file with multiple namespaces

 


Steps to parse XML files with multiple namespaces in SSIS

1. Get rid of multiple namespaces

follow the thread below, bascially apply XSLT to regenerate an XML

http://social.msdn.microsoft.com/Forums/sqlserver/en-US/2fde216c-01ab-42a5-8b43-52f5f42d35bc/multiple-namespaces-in-an-xml-document

2. Once cleaned XML has been created, follow the thread below to parse only required information using XSLT

http://blog.hoegaerden.be/2011/04/20/loading-complex-xml-using-ssis/

3. The output is the CSV file which can be inserted into tables easily

Friday, June 21, 2013

How to add new package store under Stored Packages

 

In order to create a new folder called ‘My Packages’

1. Find the file called MsDtsSrvr.ini.xml in C:\Program Files\Microsoft SQL Server\100\DTS\Binn

2. Use a notepad and add the code in underline. Also make sure folder does exist. In this case c:\SSIS does exit.

 

3. Save and make sure you restart SSIS engine via Sql Server Configuration Manager

 

<?xml version="1.0" encoding="utf-8"?>
<DtsServiceConfiguration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <StopExecutingPackagesOnShutdown>true</StopExecutingPackagesOnShutdown>
  <TopLevelFolders>
    <Folder xsi:type="SqlServerFolder">
      <Name>MSDB</Name>
      <ServerName>.</ServerName>
    </Folder>


    <Folder xsi:type="FileSystemFolder">
      <Name>File System</Name>
      <StorePath>..\Packages</StorePath>
    </Folder>


    <Folder xsi:type="FileSystemFolder">
      <Name>My Packages</Name>
      <StorePath>C:\SSIS</StorePath>
    </Folder>


  </TopLevelFolders>
</DtsServiceConfiguration>

 

image

Friday, November 9, 2012

Stupid mistake when running report project solution with more than one reports

 

I’ve created a report project solution and imported a few report rdl files. First, BIDS 2008 asked me to identify the StartItem.

Then When I try to run a report, I just click the green triangle to debugging but did not realize bids was trying to run that default report I've

picked as the StartItem. This error bothered me for a few hours until I finally figured it out .

I have to right click the report I want to run under designer and run it . Keep that in mind!!!

Friday, May 20, 2011

Dynamic connection manager in SSIS package, Use DelayValidation = TRUE on pkg level.

Since the connection manager is using the expression to make it flexible, the dataflow will always show the error icon. The tip is to turn the DelayedValidation to TRUE on the package level. In this way, the validation package won’t kick in until the run time.

Friday, March 11, 2011

ETL from Sql Server to Access , lesson learned

In destination access table, one column is Yes/No type while there is no yes/no datatype in Sql server.  Int datatype is employed where 0 indicates false and –1 indicates true.

Also in access, all varchar datatype is Unicode WT_WSTR while in Sql server, they are DT_STR. Need to use derived columns between source and target just to convert them.

Tuesday, August 24, 2010

Data type in SSIS

Source data size should be equal or less than destination data size, otherwise, it will have truncation warning.

To be continued.

Thursday, April 15, 2010

Closure of the MS BI project

will write some experience through this project
Several items that I’d like to touch are listed below:

1. SSIS tips
Development
We have 3 types of packages: daily, weekly and monthly.
Daily pkgs will pick up new records everyday and aggregate into the summary table.
For daily changed reference data, we do not pick up untill the monthly pkgs kick in.
That’s our version to Slowly Changing Dimension (ideally, SCD should be used)

weekly pkgs will truncate the fact table and bring 3 month new records

monthly pkgs will truncate dimension tables and bring all records


Deployment
So far, the deployment is using file system and import pkgs via SSMS. That’s not the best way though but DBA feels more comfortable in this way


Debugging
We have created a sysssislog table which contains all log information. If data error or pkg error occurs , dba or am will get an email immediately.

When we direct error output to another table in OLE DB destination, make sure it’s using table/view instead of fast load. If fast load is used, if the stream hits any bad data, ssis will direct all data after the bad row into the error table no matter if it’s bad or not. Be careful!

2. SSAS tips
Development
Deployment
Debugging

3. Version Control (SVN)

Tuesday, April 13, 2010

Dynamic connection manager in ssis packages

@[System::MachineName] is the right way to put in the expression of connection manager. The property is ServerName. In this way, when pkgs are executed , it will automatically use the servername which will save tons of time on the dba’s side.

 

image

Thursday, March 25, 2010

How to decipher SSIS Error Code

Here is a nice article. Have to use dtsmsg.h and use calculater to convert error code into Hex

http://blogs.msdn.com/helloworld/archive/2008/07/25/how-to-decipher-understand-ssis-error-code.aspx

Technorati Tags:

Friday, March 19, 2010

SSIS OLE DB command

 

image

This task is in the data flow task. We are trying to update a table through a series of joining with other tables. However, we do not want to output the result into a temp table and update the original table with this temp table. OLE DB command is used in this case as it can perform sql command for each row. The good side is that the original table only contains around 300 rows. So the performance is not a bit deal. If the original table contains a large volume of data, then it’s better to use output into a staging table and use the set operation to update the table.

Tuesday, March 16, 2010

How to create a time dimension in a table?

  with mycte as

(
select cast('2009-01-01' as datetime) DateValue
union all
select DateValue + 1
from    mycte  
where   DateValue + 1 < = GetDate() + 10

  )

select DateValue

from    mycte

OPTION (MAXRECURSION 0)

This script will generate date starting from 2009-01-01 to 10 days after current date.

Wednesday, March 3, 2010

Lessons learned from pilot BI project

In order to build this pilot BI project, we utilized SSIS 2008 to create the datawarehouse and used SSAS to create the OLAP cube on top of that.  In the next few blogs, I’ll post the tips and lessons learnt from the process in order to share with those who are interested and to remind myself from time to time…