SQL Online ERD/DB Modeler

[Source: http://ondras.zarovi.cz/sql/demo/]

- xp_cmdshell command (not recommended for security reasons)
- sp_start_job command

The main difference between both options is the execution method. The xp_cmdshell command is a synchronous call and the sp_start_job command is an asynchronous call.

xp_cmdshell command (synchronous)

• enable xp_cmdshell mode in Surface Area Configuration Tool for SQL Server
• use following procedure to execute SSIS package:
  
DECLARE @returncode int
EXEC @returncode = xp_cmdshell 'dtexec /f "PackageNameWithFullPath.dtsx"'


sp_start_job (asynchronous)

• define job in SQL Server Agent with SSIS execution step
• use following procedure to start SQL Server Agent job with SSIS execution step:
  
[msdb].dbo.sp_start_job @job_name='JobName'

• check execution status of job, since the call is asynchronous:
  
SELECT
[server],
[start_execution_date],
[stop_execution_date],
[run_date],
[run_duration],
[run_status],
[message]
FROM
MSDB.DBO.SYSJOBS Z
INNER JOIN
MSDB.DBO.SYSJOBACTIVITY A
ON Z.JOB_ID = A.JOB_ID
INNER JOIN
(
SELECT
MAX(SESSION_ID) AS SESSION_ID
FROM
MSDB.DBO.SYSSESSIONS
) AS B
ON A.SESSION_ID = B.SESSION_ID
LEFT JOIN
MSDB.DBO.SYSJOBHISTORY C
ON A.JOB_HISTORY_ID = C.INSTANCE_ID
WHERE
Z.NAME = 'JobName'


There is following way to make the call of sp_start_job synchronous.

sp_start_job (synchronous)

• define job in SQL Server Agent with SSIS execution step
• use following procedure to start SQL Server Agent job with SSIS execution step:
  
ALTER PROCEDURE [dbo].[AGENT_JOB_CHECK2]
-- Add the parameters for the stored procedure here
DECLARE @job_name nvarchar(100)='',
DECLARE @maxwaitmins int = 5
AS
BEGIN
set NOCOUNT ON;
set XACT_ABORT ON;
BEGIN TRY
declare @running as int
declare @seccount as int
declare @maxseccount as int
set @maxseccount = 60*@maxwaitmins
set @seccount = 0
set @running = 0
declare @job_owner sysname
declare @job_id UNIQUEIDENTIFIER
set @job_owner = SUSER_SNAME()
-- get job id
select @job_id=job_id
from msdb.dbo.sysjobs sj
where sj.name=@job_name
-- invalid job name then exit with an error
if @job_id is null
RAISERROR (N'Unknown job: %s.', 16, 1, @job_name)
-- output from stored procedure xp_sqlagent_enum_jobs is captured in the following table
declare @xp_results TABLE ( job_id UNIQUEIDENTIFIER NOT NULL,
last_run_date INT NOT NULL,
last_run_time INT NOT NULL,
next_run_date INT NOT NULL,
next_run_time INT NOT NULL,
next_run_schedule_id INT NOT NULL,
requested_to_run INT NOT NULL, -- BOOL
request_source INT NOT NULL,
request_source_id sysname COLLATE database_default NULL,
running INT NOT NULL, -- BOOL
current_step INT NOT NULL,
current_retry_attempt INT NOT NULL,
job_state INT NOT NULL)
-- start the job
declare @r as int
exec @r = msdb..sp_start_job @job_name
-- quit if unable to start
if @r<>0
RAISERROR (N'Could not start job: %s.', 16, 2, @job_name)
-- start with an initial delay to allow the job to appear in the job list (maybe I am missing something ?)
WAITFOR DELAY '0:0:10';
set @seccount = 10
-- check job run state
insert into @xp_results
execute master.dbo.xp_sqlagent_enum_jobs 1, @job_owner, @job_id
set @running= (SELECT top 1 running from @xp_results)
while @running<>0 and @seccount < @maxseccount
begin
WAITFOR DELAY '0:0:10';
set @seccount = @seccount + 10
delete from @xp_results
insert into @xp_results
execute master.dbo.xp_sqlagent_enum_jobs 1, @job_owner, @job_id
set @running= (SELECT top 1 running from @xp_results)
end
-- result: query
SELECT
[server],
[start_execution_date],
[stop_execution_date],
[run_date],
[run_duration],
[run_status], -- 0: failed, 1: success, null: running
[message]
FROM
MSDB.DBO.SYSJOBS Z
INNER JOIN
MSDB.DBO.SYSJOBACTIVITY A
ON Z.JOB_ID = A.JOB_ID
INNER JOIN
(
SELECT
MAX(SESSION_ID) AS SESSION_ID
FROM
MSDB.DBO.SYSSESSIONS
) AS B
ON A.SESSION_ID = B.SESSION_ID
LEFT JOIN
MSDB.DBO.SYSJOBHISTORY C
ON A.JOB_HISTORY_ID = C.INSTANCE_ID
WHERE Z.NAME = @job_name
-- result: not ok (=1) if still running
--if @running <> 0
--return 0
--else
--return 1
END TRY
BEGIN CATCH
DECLARE
@ErrorMessage NVARCHAR(4000),
@ErrorNumber INT,
@ErrorSeverity INT,
@ErrorState INT,
@ErrorLine INT,
@ErrorProcedure NVARCHAR(200);
SELECT
@ErrorNumber = ERROR_NUMBER(),
@ErrorSeverity = ERROR_SEVERITY(),
@ErrorState = ERROR_STATE(),
@ErrorLine = ERROR_LINE(),
@ErrorProcedure = ISNULL(ERROR_PROCEDURE(), '-');
SELECT @ErrorMessage =
N'Error %d, Level %d, State %d, Procedure %s, Line %d, ' +
'Message: '+ ERROR_MESSAGE();
RAISERROR
(
@ErrorMessage,
@ErrorSeverity,
1,
@ErrorNumber, -- original error number.
@ErrorSeverity, -- original error severity.
@ErrorState, -- original error state.
@ErrorProcedure, -- original error procedure name.
@ErrorLine -- original error line number.
);
END CATCH
END


[Source: http://blog.boxedbits.com/archives/124]

To make the problem more complex: The number of values in the “list column” is variable and there is a second column which contains corresponding values.

Excel Source

Excel Sheet

solution: SSIS Script Component

To implement a solution for the problem we use the most adequate item – the Script Component.

1. We start with a data flow task in our package and add an OLE DB Excel Source component (that returns the source data above).
   We can use following SQL statement:
   
SELECT
F1,F2,F3
FROM
[sheet1$B1:D5]

   The imported columns can be named as KEY, PERSON, DEPARTMENT.
   
   

   Excel Source

2. Then we add a Derived Column Component to the data flow as well.
   
   

   Derived Column

   In order to replace the CRs with commas we replace the imported columns with following formula:
   
REPLACE(REPLACE(REPLACE(REPLACE(PERSON,"\n",",")," ",""),",,",","),",,",",")
REPLACE(REPLACE(REPLACE(REPLACE(DEPARTMENT,"\n",",")," ",""),",,",","),",,",",")

   

   

   Derived Column Settings

3. As result we receive following list of elements:
   • – -
   • row 1
   • PERSON: Mr. Wayne,Mr. Kent,Mr. Parker
   • DEPARTMENT: Heros/Batman,Heros/Superman,Heros/Spiderman
   • – -
   • row 2
   • PERSON: Mr. Kirk,Ms. Janeway
   • DEPARTMENT: StarTrek/Classic,StartTrek/Voyager
   • – -
   • row 3
   • PERSON: “Mr. Wrong”
   • DEPARTMENT: Wrong entry 1,Wrong entry 2
   • – -
4. Now we add a Script Component to the data flow.
   When we are prompted to select the type of the component we choose Transformation.
   

   Script Component

5. Then we right-click on the Script Component and choose Edit. In the Script Transformation Editor dialog box we select all three columns to make them available as input columns. Now these column are available in the Script code.
   
   

   Script Component Columns

6. Since the tranformation is going to produce more output rows than it receives input rows we have to mark the transformation as asynchronous. This is done by setting the SynchronousInputID property of the output to None.
   
   

   Script Component Settings

7. Next, we have to specify the output columns that the output buffer will contain. So we select the transformation’s output and click Add Column and set the properties (column name and data type).
   
   

   Script Component Input / Output rows

8. Now we can add the code by clicking on the Design Script button.
   
   

   Script Component - Design Script

9. Following Code is splitting our list of items into rows:
   
' Variables
Dim i As Integer = 0
Dim itemList_PERSON As String = Row.PERSON
Dim itemList_DEPARTMENT As String = Row.DEPARTMENT
Dim delimiter As String = ','
If Not (String.IsNullOrEmpty(itemList_PERSON) Or String.IsNullOrEmpty(itemList_DEPARTMENT)) Then
Dim inputListArray_Wer() As String = itemList_PERSON.Split(New String() {delimiter}, StringSplitOptions.RemoveEmptyEntries)
Dim inputListArray_Abt() As String = itemList_DEPARTMENT.Split(New String() {delimiter}, StringSplitOptions.RemoveEmptyEntries)
If (inputListArray_Wer.Length = inputListArray_Abt.Length) Then
' new rows
For Each item As String In inputListArray_Wer
With Output0Buffer
.AddRow()
.KEY = CInt(Row.KEY)
.PERSON = item
.DEPARTMENT = inputListArray_Abt(i).ToString()
End With
' increase counter
i = i + 1
Next
Else
' Filtered Rows
With Output1Buffer
.AddRow()
.KEY = CInt(Row.KEY)
.PERSON = Row.PERSON
.DEPARTMENT = Row.DEPARTMENT
End With
End If
End If

   The code shows that there are two outputs. Whenever a record set cannot be split or the columns PERSON and DEPARTMENT have a different number of itemss the rows is redirected to the second output (here Output1).
10. In order to receive the output from the Script Component we add a Union All Component. Additionally we add two Data Viewer Components, since we want to see the resulting outputs from the Script Component.
    
    

    Union ALL

11. Running the SSIS package will bring the following result.
    
    

    Data Flow completed

12. As we can see, the first DataViewer Component show us the valid rows produced by our Script Component.
    
    

    Data Viewer valid rows

13. The second DataViewer Component show us the invalid rows produced by our Script Component.
    
    

    Data Viewer invalid rows

download
Of course I provide you the solution as download.

possible solution: T-SQL
One possible solution using T-SQL is following statement.
The function fn_Split returns all values in a list and the replace function converts the CR (carriage returns) to commas.


SELECT
[Key]
,[b.Value]
,[c.Value]
FROM
[dbo.TableExcel] AS a
CROSS APPLY
fn_Split(REPLACE(a.Persons, CHAR(10)+CHAR(13),','), ',') AS b
CROSS APPLY
fn_Split(REPLACE(a.Departments, CHAR(10)+CHAR(13),','), ',') AS c


The corresponding fn_Split function:

CREATE FUNCTION [dbo].[fn_Split](@text nvarchar(max), @delimiter char(1) = ' ')
RETURNS @Strings TABLE (
position int IDENTITY PRIMARY KEY,
value nvarchar(max)
)
AS
BEGIN
DECLARE @index int
SET @index = -1
WHILE (LEN(@text) > 0)
BEGIN
SET @index = CHARINDEX(@delimiter , @text)
IF (@index = 0) AND (LEN(@text) > 0)
BEGIN
INSERT INTO @Strings VALUES (@text)
BREAK
END
IF (@index > 1)
BEGIN
INSERT INTO @Strings
VALUES (LEFT(@text, @index - 1))
SET @text = RIGHT(@text, (LEN(@text) - @index))
END
ELSE
SET @text = RIGHT(@text, (LEN(@text) - @index))
END
RETURN
END


1. use option “IMEX=1” in ConnectionString (Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\..\MyExcel.xls;Extended Properties="Excel 8.0;HDR=Yes;IMEX=1")
2. modify registry setting “TypeGuessRows=8” to a much higher value
3. use a sql statement to retrieve data (e.g. “SELECT * FROM [sheet1$A5:M2000]“)

NOTE: The setting IMEX=1 tells the Excel driver to use Import mode. This mode reads the registry setting “ImportMixedTypes=Text” which forces mixed data to be converted to text. To achieve a more reliable column type recognition, the registry setting “TypeGuessRows=8” should be increased. By default the ISAM driver analyzes  the first eight rows and determines the column datatypes from this sampling. If the first analysed rows contain only numeric values, then setting IMEX=1 will not convert this column datatype to Text – it will remain numeric.

ATTENTION: The IMEX setting has some other modes:

• 0 is Export mode
• 1 is Import mode
• 2 is Linked mode (full update capabilities)

Registry Setting Location:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Jet\X.X\Engines\Excel


supplement
If you experience following error the above described solution could fix it.
1. An OLE DB error has occurred. Error code: 0x80040E21.
2. Failed to retrieve long data for column “XYZ”.
3. There was an error with output column “XYZ” (55488) on output “Excel Source Output” (109). The column status returned was: “DBSTATUS_UNAVAILABLE”.
4. The “output column “XYZ” (55488)” failed because error code 0xC0209071 occurred, and the error row disposition on “output column “XYZ” (55488)” specifies failure on error. An error occurred on the specified object of the specified component.
5. [DTS.Pipeline] Error: The PrimeOutput method on component “Info Source” (101) returned error code 0xC0209029. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
6. [DTS.Pipeline] Error: Thread “SourceThread0″ has exited with error code 0xC0047038.

Just run following command within Management Studio:
SELECT
@@VERSION,
SERVERPROPERTY('productversion'),
SERVERPROPERTY ('productlevel'),
SERVERPROPERTY ('edition')

As result you receive something like:

Microsoft SQL Server 2005 - ... Windows NT 5.2 (Build 3790: SP 2)
  9.00.4035.00
  SP3
  Standard Edition (64-bit)

INSERT INTO table1 (column1, column2)
VALUES(@parameter1, (SELECT columnX FROM table2
WHERE columnY = '@parameter2'))

oder

INSERT INTO table1 (column1, column2)
SELECT @parameter1, columnX FROM table2
WHERE columnY = '@parameter2'

Quelle: [SQLTeam] und [Ive]

In Queries ist es jedoch etwas schwierig diese Optionen abzufragen. Folgende VBA-Funktion schafft da Abhilfe:

Function InMultiSelect(frms, ctrl As String, col As Integer, data As Variant, ParamArray OtherArgs()) As Boolean
'Checks whether a Variant (data or OtherArgs) is included in the specified column (col) of a ListBox (ctrl)
'in a certain Form (frms)
On Error GoTo Error_InMultiSelect
Dim varItm As Variant
Dim index As Integer
Dim ctl As Control
Dim frm As Form
Set frm = Forms(frms)
Set ctl = frm.Controls(ctrl)
InMultiSelect = False
For Each varItm In ctl.ItemsSelected
If InMultiSelect = True Then Exit For
If CStr(data) = CStr(ctl.Column(col, varItm)) Then InMultiSelect = True
For index = LBound(OtherArgs) To UBound(OtherArgs)
If InMultiSelect = True Then Exit For
If CStr(OtherArgs(index)) = CStr(ctl.Column(col, varItm)) Then InMultiSelect = True
Next index
Next varItm
Exit Function
Error_InMultiSelect:
InMultiSelect = False
Exit Function
End Function


In den Abfragen (Queries) selbst wird diese Funktion dann beispielsweise so aufgerufen:


SELECT *
FROM [tblTabelle]
WHERE InMultiSelect("[frmFormMitMultiSelectAuswahl]","[lstListeMitMultiSelectAuswahl]",0,[tblTabelle].[Spalte]))<>False);


Dies wird gerade wenn man mit temporären Tabellen arbeitet zum Problem.

Eine Abhilfe liefert folgende VBA-Funktion:

' check if a table exists
Function tableExists(tableName As String) As Boolean
On Error GoTo Error_tableExists
Dim strTableName
' assign tableName to String
strTableName = CurrentDb.TableDefs(tableName)
' if no error occurs then set tableExists to true
tableExists = True
Exit_tableExists:
On Error Resume Next
Exit Function
Error_tableExists:
Select Case Err.Number
Case 3265 'Item not found in this collection
tableExists = False
Resume Exit_tableExists
Case Else
MsgBox "Error " & Err.Number & ": " & Err.Description, vbCritical
Resume Exit_tableExists
End Select
End Function