tag:blogger.com,1999:blog-9549873290162014132024-03-13T15:33:21.735+05:30 All about SQLServerwritings on my experiences and learnings in SQL Server...Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.comBlogger296125tag:blogger.com,1999:blog-954987329016201413.post-72039137678021430462020-03-10T03:10:00.002+05:302020-03-10T03:17:28.542+05:30How to use R script in Power BI<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, I have embedded the video link to my you tube channel and in the video I have showed how to use R script in Power BI. I have also posted below the scripts shown in the video.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In the video, I have explained:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">1)How to R script in Power BI</span><br />
<span style="font-family: arial, helvetica, sans-serif;">2)Example dataset - COVID-19 (2019-nCoV) github data repository by Johns Hopkins University. How to invoke this data dynamically using Rcurl.</span><br />
<span style="font-family: arial, helvetica, sans-serif;">3)How to configure to specific R version in R Studio & Power BI</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: arial, helvetica, sans-serif;">4)Sample visualization in Power BI using R dataset</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://youtu.be/srKZthMOLlA" target="_blank">How to use R script in Power BI</a></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">--To enable external scripts execution</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">EXEC sp_configure 'external scripts enabled', 1</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">RECONFIGURE WITH OVERRIDE</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">--To print R version</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">EXECUTE sp_execute_external_script @language = N'R'</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> , @script = N'print(version)';</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">GO</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">--To list all R packages installed</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">EXEC sp_execute_external_script @language = N'R'</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> , @script = N'</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">OutputDataSet <- data.frame(installed.packages()[,c("Package", "Version", "Depends", "License", "LibPath")]);'</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">WITH result sets((</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> Package NVARCHAR(255)</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> , Version NVARCHAR(100)</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> , Depends NVARCHAR(4000)</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> , License NVARCHAR(1000)</span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> , LibPath NVARCHAR(2000)</span><br />
<span style="background-color: #0b5394;"><span style="color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"></span></span><br />
<span style="background-color: #0b5394; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"> ));</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u>R Script used in power BI:</u></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">library(RCurl)</span><br />
<span style="background-color: #134f5c; color: #eeeeee;"><br /></span>
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">dt <- format(as.Date(Sys.time())-1, "%m-%d-%Y")</span><br />
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">uri1 <- "https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports/"</span><br />
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">uri <- paste0(uri1,dt,".csv")</span><br />
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">rawdata <- getURL(uri)</span><br />
<span style="background-color: #134f5c;"><span style="color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;"></span></span><br />
<span style="background-color: #134f5c; color: #eeeeee; font-family: "arial" , "helvetica" , sans-serif;">convidds <- read.csv(text = rawdata)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>See Also:</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://www.allaboutmssql.com/p/microsoft-excel-powerpoint-and-power-bi.html" target="_blank">My other posts on Power BI</a></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_59.html" target="_blank">SQL Server Import/Export excel using R</a></i></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<br />
<br />
<br /></div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-77689721717136340962019-07-06T12:36:00.002+05:302019-07-06T12:47:13.419+05:30SQL Server Import/Export to Excel using Python script<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="https://www.blogger.com/null" name="Introduction" style="color: #00749e; cursor: pointer; font-size: 36.4px; font-variant-east-asian: normal; font-variant-numeric: normal;"></a><span style="color: #707070; font-family: "segoe ui light" , "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 36.4px; line-height: 47.32px;">Introduction</span><br />
<span style="color: #707070; font-family: "segoe ui light" , "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 36.4px; line-height: 47.32px;"><br /></span>
<span style="color: #2a2a2a;"><span style="font-family: "arial" , "helvetica" , sans-serif;">In previous article, we saw how to <a href="https://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_59.html" target="_blank"><i>import/export excel to/from SQL Server by executing R script within T-SQL</i></a>. </span></span><br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">
In this post, let us see another similar approach to import excel into SQL Server and export SQL server data to excel by executing Python script within T-SQL.
</span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
There are some existing methods to do this using BCP, Bulk Insert, Import & Export wizard from SSMS, SSIS, Azure data factory, Linked server & OPENROWSET query and SQLCMD.<br />
<br />
BULK INSERT statement, the BCP tool, or Azure Data Factory <a href="https://docs.microsoft.com/en-us/sql/relational-databases/import-export/import-data-from-excel-to-sql?view=sql-server-2017" style="color: #00749e; cursor: pointer; outline-style: none; outline-width: 0px; text-decoration-line: none;" target="_blank">
can't read Excel files directly <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: 10px; max-width: 640px; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a> <br />
BCP - Workaround has to be done to include the header while exporting<br />
SSIS - Though it supports exporting to excel, with dynamic source & destination, handling mapping between source to target increases the complexity of the package<br />
SQLCMD - Cannot export output in Excel file format</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: #2a2a2a; line-height: 16.98px;">
</span><span style="color: #2a2a2a; line-height: 16.98px;">
</span></span><br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
R & Python language extension was introduced in SQL Server 2016 & 2017 as part of machine learning. With support of
<a href="https://blog.revolutionanalytics.com/2018/11/r-support-in-azure-sql-database.html" style="color: #00749e; cursor: pointer; outline-style: none; outline-width: 0px; text-decoration-line: none;" target="_blank">
R in Azure SQL database <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: 10px; max-width: 640px; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a> and <a href="https://docs.microsoft.com/en-us/sql/language-extensions/language-extensions-overview?view=sqlallproducts-allversions" style="color: #00749e; cursor: pointer; outline-style: none; outline-width: 0px; text-decoration-line: none;" target="_blank">
Java language extension support in SQL Server 2019 <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: 10px; max-width: 640px; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a>, this new approach can be used extensively as it easy, fast and flexible.</span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<br />
<a href="https://www.blogger.com/null" name="Pre-requisites" style="color: #00749e; cursor: pointer; font-size: 36.4px; font-variant-east-asian: normal; font-variant-numeric: normal;"></a><span style="color: #707070; font-family: "segoe ui light" , "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 36.4px; line-height: 47.32px;">Pre-requisites</span><br />
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">We have used <a href="https://www.microsoft.com/en-us/sql-server/sql-server-2019" style="color: #00749e; cursor: pointer; outline-style: none; outline-width: 0px; text-decoration-line: none;" target="_blank">
SQL Server 2019 evaluation edition <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: 10px; max-width: 640px; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a> on Windows 10 64 bit and <a href="https://cloudblogs.microsoft.com/sqlserver/2016/06/09/wideworldimporters-the-new-sql-server-sample-database/" style="color: #00749e; cursor: pointer; outline-style: none; outline-width: 0px; text-decoration-line: none;" target="_blank">
WideWorldImporters SQL Server sample database <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: 10px; max-width: 640px; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a> for this example.</span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
Below scripts will work starting from SQL Server 2017 and above (as execution of Python language using T-SQL was introduced in SQL Server 2017).</span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
Only pre-requisite step is to <a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/install/sql-machine-learning-services-windows-install?view=sql-server-2017" style="color: #00749e; cursor: pointer; outline-style: none; outline-width: 0px; text-decoration-line: none;" target="_blank">
install Python services and then from SSMS enable the external scripting feature. Restart the database engine and then verify the installation as mentioned in MSDN <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: 10px; max-width: 640px; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a>.</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: #2a2a2a; line-height: 16.98px;">
</span><span style="color: #2a2a2a; line-height: 16.98px;">
</span><span style="color: #2a2a2a; line-height: 16.98px;">
</span></span><br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
Provide folder permission to access the excel files during import / export process, right-click on folder -> Properties -> Security -> (Full Control) to "ALL_APPLICATION_PACKAGES".</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">pandas.DataFrame.to_excel & pandas.read_excel are used to export and import excel which are installed by default<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can check that by running below statement in SSMS:</span></div>
<br />
<div style="background-color: white; color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal;">
<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="color: #006699; font-weight: 700;">EXECUTE</code>
<code style="color: black;">sp_execute_external_script</code></span></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal;">
<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="color: black;">@language =N</code><code style="color: blue;">'Python'</code><code style="color: black;">,</code></span></div>
<div style="background-color: white; color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal;">
<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="color: black;">@script=N</code><code style="color: blue;">'import pip</code></span></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal;">
<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="color: blue;">for i in pip.get_installed_distributions():</code></span></div>
<div style="background-color: white; color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal;">
<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"> </code><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="color: blue;">print(i)'</code><code style="color: black;">;</code></span></span></div>
<span style="color: #2a2a2a; font-family: "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 12.13px;">
</span><span style="color: #2a2a2a; font-family: "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 12.13px;">
</span><span style="color: #2a2a2a; font-family: "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 12.13px;">
</span><span style="color: #2a2a2a; font-family: "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 12.13px;">
</span><span style="color: #2a2a2a; font-family: "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 12.13px;">
</span><br />
<div style="background-color: #f8f8f8; color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal;">
<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><code style="color: black;">GO</code></span></div>
<br />
<br />
<h1 style="clear: both; color: #707070; font-size: 36.4px; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 100; line-height: 47.32px;">
<a href="https://www.blogger.com/null" name="Exporting_SQL_Server_data_list_of_tables_views_into_Excel_file" style="color: #00749e; cursor: pointer;"></a>Exporting SQL Server data (list of tables, views) into Excel file</h1>
<span style="color: #2a2a2a; font-family: "segoe ui" , "lucida grande" , "verdana" , "arial" , "helvetica" , sans-serif; font-size: 12.13px; line-height: 16.98px;">
</span>
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
For this approach, we have created a Stored procedure named "usp_ExportExcel" in "WideWorldImporters" database. Executing the Stored procedure based on input parameters exports SQL Server data to excel files<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
Stored procedure - Has below three input parameters and writes the output to excel file<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span><br />
<table style="font-size: 12.13px; margin-left: 1px;">
<tbody style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Parameter</span></span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Description</span></span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">@ExportPath</span> </span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">P<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">ath for exporting excel files</span> </span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">@SchemaName</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">L<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">ist of objects under this schema to be exported. Can have multiple values separated by comma</span> </span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">@ObjectlisttoExport</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">L<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">ist of tables, views to be exported. Can have multiple values separated by comma</span></span></td></tr>
</tbody></table>
</div>
<br />
<br />
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: grey;">OR</code> <code style="color: #006699; font-weight: bold;">ALTER</code> <code>PROC usp_ExportExcel (@ExportPath NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@SchemaName NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),</code></div>
<div style="background-color: white;">
<code> </code><code>@ObjectlisttoExport NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>)</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">AS</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">SET</code> <code>NOCOUNT </code><code style="color: #006699; font-weight: bold;">ON</code><code>;</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">BEGIN</code> <code>TRY</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code>IF </code><code style="color: deeppink;">ISNULL</code><code>(@ExportPath,</code><code style="color: blue;">''</code><code>) <> </code><code style="color: blue;">''</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>@ExportPath = </code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code style="color: deeppink;">RIGHT</code><code>(@ExportPath,1) = </code><code style="color: blue;">'\' THEN @ExportPath ELSE CONCAT(@ExportPath,'</code><code>\</code><code style="color: blue;">') END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">DECLARE @ValidPath TABLE (ValidPathCheck BIT)</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">INSERT @ValidPath</code></div>
<div style="background-color: white;">
<code style="color: blue;">EXEC sp_execute_external_script</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">@language =N'</code><code>Python</code><code style="color: blue;">',</code></div>
<div style="background-color: white;">
<code style="color: blue;">@script=N'</code></div>
<div style="background-color: #f8f8f8;">
<code>import pandas </code><code style="color: #006699; font-weight: bold;">as</code> <code>pd</code></div>
<div style="background-color: white;">
<code>d = os.path.isdir(ExportFilePath) </code></div>
<div style="background-color: #f8f8f8;">
<code>OutputDataSet = pd.DataFrame(<a class="internal-link new-page" href="https://social.technet.microsoft.com/wiki/contents/articles/add.aspx?WikiParentPageID=53063&WikiPageKey=d" title="Click to create new page titled: d">d</a>,columns=[</code><code style="color: blue;">"Filename"</code><code>])</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
<code style="color: blue;">,@params = N'</code><code>@ExportFilePath NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>)</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">,@ExportFilePath = @ExportPath</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">IF (SELECT ValidPathCheck FROM @ValidPath) = 1</code></div>
<div style="background-color: white;">
<code style="color: blue;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">IF ISNULL(@SchemaName,'</code><code style="color: blue;">') <> '</code><code style="color: blue;">' OR ISNULL(@ObjectlisttoExport,'</code><code style="color: blue;">') <> '</code><code style="color: blue;">' </code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: blue;">DROP TABLE IF EXISTS #ExportTablesList, #FinalExportList</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: blue;">CREATE TABLE #ExportTablesList(TableCount INT IDENTITY(1,1),Cols NVARCHAR(MAX),TableName NVARCHAR(200))</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">--Get the list of objects to be exported </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">INSERT #ExportTablesList (Cols,TableName)</code></div>
<div style="background-color: white;">
<code style="color: blue;">SELECT CASE WHEN TY.name IN ('</code><code style="color: #006699; font-weight: bold;">date</code><code style="color: blue;">','</code><code>datetime2</code><code style="color: blue;">','</code><code>datetimeoffset</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">time</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">timestamp</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">decimal</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">bit</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">int</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">bigint</code><code style="color: blue;">') </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">THEN CONCAT('</code><code>TRY_CONVERT(</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">VARCHAR</code><code>(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),</code><code style="color: blue;">',C.name,'</code><code>) </code><code style="color: #006699; font-weight: bold;">AS</code> <code style="color: blue;">',QUOTENAME(C.NAME)) </code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">ELSE C.name END Cols -- To cover poor data type conversions b/n Python & SQL Server</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,CONCAT(SCHEMA_NAME(T.SCHEMA_ID),'</code><code>.</code><code style="color: blue;">',T.name) TableName</code></div>
<div style="background-color: white;">
<code style="color: blue;">FROM Sys.tables T</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">JOIN sys.columns C</code></div>
<div style="background-color: white;">
<code style="color: blue;">ON T.object_id = C.object_id</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">JOIN sys.types TY </code></div>
<div style="background-color: white;">
<code style="color: blue;">ON C.[user_type_id] = TY.[user_type_id]</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">WHERE Schema_name(T.schema_id) IN (SELECT value FROM STRING_SPLIT(@SchemaName, '</code><code>,</code><code style="color: blue;">')) </code></div>
<div style="background-color: white;">
<code style="color: blue;">-- Ignore the datatypes that are not required to be exported</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">AND TY.name NOT IN ('</code><code>geography</code><code style="color: blue;">','</code><code>varbinary</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">binary</code><code style="color: blue;">','</code><code>text</code><code style="color: blue;">', '</code><code>ntext</code><code style="color: blue;">', '</code><code>image</code><code style="color: blue;">', '</code><code>hierarchyid</code><code style="color: blue;">', '</code><code>xml</code><code style="color: blue;">', '</code><code>sql_variant</code><code style="color: blue;">') </code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">INSERT #ExportTablesList (Cols,TableName)</code></div>
<div style="background-color: white;">
<code style="color: blue;">SELECT CASE WHEN TY.name IN ('</code><code style="color: #006699; font-weight: bold;">date</code><code style="color: blue;">','</code><code>datetime2</code><code style="color: blue;">','</code><code>datetimeoffset</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">time</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">timestamp</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">decimal</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">bit</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">int</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">bigint</code><code style="color: blue;">') </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">THEN CONCAT('</code><code>TRY_CONVERT(</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">VARCHAR</code><code>(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),</code><code style="color: blue;">',C.name,'</code><code>) </code><code style="color: #006699; font-weight: bold;">AS</code> <code style="color: blue;">',QUOTENAME(C.NAME)) </code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">ELSE C.name END Cols -- To cover poor data type conversions b/n Python & SQL Server</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,CONCAT(SCHEMA_NAME(T.SCHEMA_ID),'</code><code>.</code><code style="color: blue;">',T.name) TableName</code></div>
<div style="background-color: white;">
<code style="color: blue;">FROM Sys.tables T</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">JOIN sys.columns C</code></div>
<div style="background-color: white;">
<code style="color: blue;">ON T.object_id = C.object_id</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">JOIN sys.types TY </code></div>
<div style="background-color: white;">
<code style="color: blue;">ON C.[user_type_id] = TY.[user_type_id] </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">WHERE CONCAT(SCHEMA_NAME(T.SCHEMA_ID),'</code><code>.</code><code style="color: blue;">',T.name) IN (SELECT value FROM STRING_SPLIT(@ObjectlisttoExport, '</code><code>,</code><code style="color: blue;">'))</code></div>
<div style="background-color: white;">
<code style="color: blue;">-- Ignore the datatypes that are not required to be exported</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">AND TY.name NOT IN ('</code><code>geography</code><code style="color: blue;">','</code><code>varbinary</code><code style="color: blue;">','</code><code style="color: #006699; font-weight: bold;">binary</code><code style="color: blue;">','</code><code>text</code><code style="color: blue;">', '</code><code>ntext</code><code style="color: blue;">', '</code><code>image</code><code style="color: blue;">', '</code><code>hierarchyid</code><code style="color: blue;">', '</code><code>xml</code><code style="color: blue;">', '</code><code>sql_variant</code><code style="color: blue;">') </code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">--Dedup of object list</code></div>
<div style="background-color: white;">
<code style="color: blue;">;WITH dedup</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">AS</code></div>
<div style="background-color: white;">
<code style="color: blue;">(</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">SELECT *,ROW_NUMBER()OVER(PARTITION BY TableName,Cols ORDER BY Cols) Rn FROM #ExportTablesList</code></div>
<div style="background-color: white;">
<code style="color: blue;">)</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">DELETE FROM dedup</code></div>
<div style="background-color: white;">
<code style="color: blue;">WHERE Rn > 1</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">--Forming columns list as comma separated </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">SELECT TableName,IDENTITY(INT,1,1) AS TableCount</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">, STUFF(</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">(</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">SELECT '</code><code>, </code><code style="color: blue;">' + C.Cols</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">From #ExportTablesList As C</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">WHERE C.TableName = T.TableName</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">FOR XML PATH('</code><code style="color: blue;">')</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">), 1, 2, '</code><code style="color: blue;">') AS Cols</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">INTO #FinalExportList</code></div>
<div style="background-color: white;">
<code style="color: blue;">From #ExportTablesList As T</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">GROUP BY TableName</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: blue;">DECLARE @I INT = 1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@TableName NVARCHAR(200)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@SQL NVARCHAR(MAX) = N'</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@PythonScript NVARCHAR(MAX) = N'</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@ExportFilePath NVARCHAR(MAX) = N'</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">--Loop through the object list to export as excel </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">WHILE @I <= (SELECT COUNT(TableName) FROM #FinalExportList)</code></div>
<div style="background-color: white;">
<code style="color: blue;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">-- Just for testing purpose top 10 records are selected</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">SELECT @SQL = CONCAT('</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">TOP</code> <code>10 </code><code style="color: blue;">',Cols,'</code> <code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: blue;">',TableName,'</code><code>;</code><code style="color: blue;">')</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@TableName = TableName</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">FROM #FinalExportList WHERE TableCount = @I</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: blue;">SET @PythonScript = N'</code></div>
<div style="background-color: #f8f8f8;">
<code>FullFilePath = ExcelFilePath+TableName+</code><code style="color: blue;">".xlsx"</code></div>
<div style="background-color: white;">
<code>InputDataSet.to_excel(FullFilePath,sheet_name=TableName.split(</code><code style="color: blue;">"."</code><code>)[-1],</code><code style="color: #006699; font-weight: bold;">index</code><code>=</code><code style="color: #006699; font-weight: bold;">False</code><code>)</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: blue;">EXEC sp_execute_external_script</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">@language = N'</code><code>Python</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@script = @PythonScript</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@input_data_1 = @SQL</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@params = N'</code><code>@ExcelFilePath NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>), @TableName NVARCHAR(200)</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@ExcelFilePath = @ExportPath -- file path where Excel files are placed</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@TableName = @TableName</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">SET @I = @I + 1</code></div>
<div style="background-color: white;">
<code style="color: blue;">END</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END ELSE PRINT '</code><code style="color: #006699; font-weight: bold;">Schema</code> <code style="color: #006699; font-weight: bold;">name</code> <code style="color: #006699; font-weight: bold;">of</code> <code>objects </code><code style="color: grey;">or</code> <code>list </code><code style="color: #006699; font-weight: bold;">of</code> <code>objects (separated </code><code style="color: #006699; font-weight: bold;">by</code> <code>comma) </code><code style="color: #006699; font-weight: bold;">to</code> <code>be exported need </code><code style="color: #006699; font-weight: bold;">to</code> <code>be mentioned</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">END ELSE PRINT '</code><code>Invalid folder path</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END ELSE PRINT '</code><code>Export folder path need </code><code style="color: #006699; font-weight: bold;">to</code> <code>be mentioned</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END TRY</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">BEGIN CATCH</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">PRINT '</code><code>Issue while executing this SP, please </code><code style="color: #006699; font-weight: bold;">check</code> <code>whether there </code><code style="color: #006699; font-weight: bold;">is</code> <code>permission </code><code style="color: #006699; font-weight: bold;">to</code> <code style="color: #006699; font-weight: bold;">execute</code> <code>the script / </code><code style="color: #006699; font-weight: bold;">to</code> <code>access the folder </code><code style="color: grey;">and</code> <code>input params are valid'</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">END</code> <code>CATCH</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">END</code></div>
<br />
<br />
<br />
<br />
<h1 style="clear: both; color: #707070; font-size: 36.4px; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 100; line-height: 47.32px;">
<a href="https://www.blogger.com/null" name="Importing_Excel_file_into_SQL_Server" style="color: #00749e; cursor: pointer;"></a>Importing Excel file into SQL Server</h1>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
For this approach, we have created a Stored procedure named "usp_ImportExcel" in <span style="border: 0px none rgb(42, 42, 42); line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
"WideWorldImporters" </span>database. Executing the Stored procedure based on input parameters imports excel files to SQL Server table<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: #2a2a2a; line-height: 16.98px;">
</span><span style="color: #2a2a2a; line-height: 16.98px;">
</span></span><br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Stored procedure - Has below six input parameters <br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span><br />
<table style="margin-left: 1px;">
<tbody style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Parameter</span></span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Description</span></span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> @ImportPath</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white; color: #b03200;"></span><span style="color: #002000;">Path where excel files are placed for importing into SQL Server</span><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"> </span><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> @DBConnectionString</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Target SQL Server database connection string where files are imported. Can be<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
Database=DB name; Trusted_Connection=True<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
or<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
Database=<span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">DB name</span>;Uid= user name;Pwd=Password<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> @ImportAll</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">If set to 1 then all files in the mentioned path are imported. If set to 0 then only mentioned files are imported</span></span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> @CombineTarget</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Flag to decide single target table <span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">for each source file</span> (files with same structure) or separate target table for each source file </span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> @ExcelFileName</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">If @ImportAll = 0 then excel file name needs to be passed to this parameter</span></span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> @ExcelSheetName</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">If @ImportAll = 0 then corresponding sheet name of the excel file needs to be passed to this parameter</span> </span></td>
</tr>
</tbody>
</table>
<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: grey;">OR</code> <code style="color: #006699; font-weight: bold;">ALTER</code> <code style="color: black;">PROC usp_ImportExcel (@ImportPath NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code style="color: black;">),</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">@DBConnectionString NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code style="color: black;">),</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">@ImportAll </code><code style="color: #006699; font-weight: bold;">BIT</code><code style="color: black;">,</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">@CombineTarget </code><code style="color: #006699; font-weight: bold;">BIT</code><code style="color: black;">,</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">@ExcelFileName NVARCHAR(200),</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">@ExcelSheetName NVARCHAR(50)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">AS</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SET</code> <code style="color: black;">NOCOUNT </code><code style="color: #006699; font-weight: bold;">ON</code><code style="color: black;">;</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">BEGIN</code> <code style="color: black;">TRY</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">IF </code><code style="color: deeppink;">ISNULL</code><code style="color: black;">(@ImportPath,</code><code style="color: blue;">''</code><code style="color: black;">) <> </code><code style="color: blue;">''</code> <code style="color: grey;">AND</code> <code style="color: deeppink;">ISNULL</code><code style="color: black;">(@DBConnectionString,</code><code style="color: blue;">''</code><code style="color: black;">) <> </code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">@ImportPath = </code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code style="color: deeppink;">RIGHT</code><code style="color: black;">(@ImportPath,1) = </code><code style="color: blue;">'\' THEN @ImportPath ELSE CONCAT(@ImportPath,'</code><code style="color: black;">\</code><code style="color: blue;">') END</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">DECLARE @Serv NVARCHAR(200) = CONCAT(CHAR(39),CHAR(39),@@SERVERNAME,CHAR(39),CHAR(39))</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">DECLARE @ValidPath TABLE (ValidPathCheck BIT)</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">INSERT @ValidPath</code></div>
<div style="background-color: white;">
<code style="color: blue;">EXEC sp_execute_external_script</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">@language =N'</code><code style="color: black;">Python</code><code style="color: blue;">',</code></div>
<div style="background-color: white;">
<code style="color: blue;">@script=N'</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">import pandas </code><code style="color: #006699; font-weight: bold;">as</code> <code style="color: black;">pd</code></div>
<div style="background-color: white;">
<code style="color: black;">d = os.path.isdir(ImportFilePath) </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">OutputDataSet = pd.DataFrame(<a class="internal-link new-page" href="https://social.technet.microsoft.com/wiki/contents/articles/add.aspx?WikiParentPageID=53063&WikiPageKey=d" title="Click to create new page titled: d">d</a>,columns=[</code><code style="color: blue;">"Filename"</code><code style="color: black;">])</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
<code style="color: blue;">,@params = N'</code><code style="color: black;">@ImportFilePath NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code style="color: black;">)</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">,@ImportFilePath = @ImportPath</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">IF (SELECT ValidPathCheck FROM @ValidPath) = 1</code></div>
<div style="background-color: white;">
<code style="color: blue;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">IF (@ImportAll = 0 AND (ISNULL(@ExcelFileName,'</code><code style="color: blue;">') <> '</code><code style="color: blue;">' AND ISNULL(@ExcelSheetName,'</code><code style="color: blue;">') <> '</code><code style="color: blue;">')) </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">OR (@ImportAll = 1 AND (ISNULL(@ExcelFileName,'</code><code style="color: blue;">') = '</code><code style="color: blue;">' AND ISNULL(@ExcelSheetName,'</code><code style="color: blue;">') = '</code><code style="color: blue;">'))</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">BEGIN</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: blue;">DECLARE @PythonScript NVARCHAR(MAX) =CONCAT('</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">import pandas </code><code style="color: #006699; font-weight: bold;">as</code> <code style="color: black;">pd</code></div>
<div style="background-color: white;">
<code style="color: black;">import os</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">import glob</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">from</code> <code style="color: black;">revoscalepy import RxSqlServerData, rx_data_step</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">sqlConnString = </code><code style="color: blue;">"Driver=SQL Server;Server=Serv; ',@DBConnectionString,'"</code></div>
<div style="background-color: white;">
<code style="color: black;">Filefolderepath = ImportFilePath+</code><code style="color: blue;">"*.xlsx"</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">if ImportAll ==1 </code><code style="color: grey;">and</code> <code style="color: black;">CombineTarget==0:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">FullFilePath </code><code style="color: grey;">in</code> <code style="color: black;">glob.glob(Filefolderepath):</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">Filename = os.path.basename(FullFilePath).</code><code style="color: deeppink;">replace</code><code style="color: black;">(</code><code style="color: blue;">".xlsx"</code><code style="color: black;">,</code><code style="color: blue;">""</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">xl = pd.ExcelFile(FullFilePath)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">sheetname </code><code style="color: grey;">in</code> <code style="color: black;">xl.sheet_names:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">Output</code> <code style="color: black;">= pd.read_excel(FullFilePath, sheetname=sheetname, na_filter=</code><code style="color: #006699; font-weight: bold;">False</code><code style="color: black;">).astype(str)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">if </code><code style="color: grey;">not</code> <code style="color: #006699; font-weight: bold;">Output</code><code style="color: black;">.empty:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">sqlDS = RxSqlServerData(connection_string = sqlConnString,</code><code style="color: #006699; font-weight: bold;">table</code> <code style="color: black;">= </code><code style="color: blue;">""</code><code style="color: black;">.</code><code style="color: grey;">join</code><code style="color: black;">(fl </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">fl </code><code style="color: grey;">in</code> <code style="color: black;">Filename if fl.isalnum())+</code><code style="color: blue;">"_"</code><code style="color: black;">+</code><code style="color: blue;">""</code><code style="color: black;">.</code><code style="color: grey;">join</code><code style="color: black;">(sh </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">sh </code><code style="color: grey;">in</code> <code style="color: black;">sheetname if sh.isalnum()))</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">rx_data_step(input_data = </code><code style="color: #006699; font-weight: bold;">Output</code><code style="color: black;">, output_file = sqlDS,overwrite = </code><code style="color: #006699; font-weight: bold;">True</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code style="color: black;">if ImportAll ==1 </code><code style="color: grey;">and</code> <code style="color: black;">CombineTarget==1:</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">df2=pd.DataFrame()</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">FullFilePath </code><code style="color: grey;">in</code> <code style="color: black;">glob.glob(Filefolderepath):</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">Filename = os.path.basename(FullFilePath).</code><code style="color: deeppink;">replace</code><code style="color: black;">(</code><code style="color: blue;">".xlsx"</code><code style="color: black;">,</code><code style="color: blue;">""</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">xl = pd.ExcelFile(FullFilePath)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">sheetname </code><code style="color: grey;">in</code> <code style="color: black;">xl.sheet_names:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">Output</code> <code style="color: black;">= pd.read_excel(FullFilePath, sheetname=sheetname).columns.astype(str)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">Output</code> <code style="color: black;">= </code><code style="color: blue;">","</code><code style="color: black;">.</code><code style="color: grey;">join</code><code style="color: black;">(list(</code><code style="color: #006699; font-weight: bold;">Output</code><code style="color: black;">))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">df1 = pd.DataFrame([[Filename,sheetname,FullFilePath,</code><code style="color: #006699; font-weight: bold;">Output</code><code style="color: black;">]],columns=[</code><code style="color: blue;">"Filename"</code><code style="color: black;">,</code><code style="color: blue;">"sheetname"</code><code style="color: black;">,</code><code style="color: blue;">"FullFilePath"</code><code style="color: black;">,</code><code style="color: blue;">"Headers"</code><code style="color: black;">])</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">df2=df2.append(df1,ignore_index=</code><code style="color: #006699; font-weight: bold;">True</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">sqlDS = RxSqlServerData(connection_string = sqlConnString,</code><code style="color: #006699; font-weight: bold;">table</code> <code style="color: black;">= </code><code style="color: blue;">"Tbl_PyImpExp1"</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">rx_data_step(input_data = df2, output_file = sqlDS,overwrite = </code><code style="color: #006699; font-weight: bold;">True</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code style="color: black;">if ImportAll ==0:</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">Filename =ImportFilePath+ExcelFileName+</code><code style="color: blue;">".xlsx"</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">exists = os.path.isfile(Filename)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">if exists </code><code style="color: grey;">and</code> <code style="color: black;">ExcelSheetName </code><code style="color: grey;">in</code> <code style="color: black;">pd.ExcelFile(Filename).sheet_names:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">Output</code> <code style="color: black;">= pd.read_excel(Filename, sheetname=ExcelSheetName, na_filter=</code><code style="color: #006699; font-weight: bold;">False</code><code style="color: black;">).astype(str)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">if </code><code style="color: grey;">not</code> <code style="color: #006699; font-weight: bold;">Output</code><code style="color: black;">.empty:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">sqlDS = RxSqlServerData(connection_string = sqlConnString,</code><code style="color: #006699; font-weight: bold;">table</code> <code style="color: black;">= </code><code style="color: blue;">""</code><code style="color: black;">.</code><code style="color: grey;">join</code><code style="color: black;">(fl </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">fl </code><code style="color: grey;">in</code> <code style="color: black;">ExcelFileName if fl.isalnum())+</code><code style="color: blue;">"_"</code><code style="color: black;">+</code><code style="color: blue;">""</code><code style="color: black;">.</code><code style="color: grey;">join</code><code style="color: black;">(sh </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">sh </code><code style="color: grey;">in</code> <code style="color: black;">ExcelSheetName if sh.isalnum()))</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">rx_data_step(input_data = </code><code style="color: #006699; font-weight: bold;">Output</code><code style="color: black;">, output_file = sqlDS,overwrite = </code><code style="color: #006699; font-weight: bold;">True</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">else</code><code style="color: black;">:</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">print(</code><code style="color: blue;">"Invalid Excel file or sheet name"</code><code style="color: black;">)</code><code style="color: blue;">')</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">EXEC sp_execute_external_script</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">@language = N'</code><code style="color: black;">Python</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@script = @PythonScript</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@params = N'</code><code style="color: black;">@ImportFilePath NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code style="color: black;">),@ImportAll </code><code style="color: #006699; font-weight: bold;">BIT</code><code style="color: black;">,@CombineTarget </code><code style="color: #006699; font-weight: bold;">BIT</code><code style="color: black;">,@ExcelFileName NVARCHAR(200),@ExcelSheetName NVARCHAR(50),@Serv NVARCHAR(200)</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@ImportFilePath = @ImportPath</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@ImportAll = @ImportAll</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@CombineTarget = @CombineTarget</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@ExcelFileName = @ExcelFileName</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@ExcelSheetName = @ExcelSheetName</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@Serv = @Serv</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">IF @ImportAll =1 AND @CombineTarget =1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">IF OBJECT_ID('</code><code style="color: black;">Tbl_PyImpExp1</code><code style="color: blue;">') IS NOT NULL</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">DROP TABLE IF EXISTS Tbl_PyImpExp2</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">;WITH FileList</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">As(</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">SELECT [Filename]</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,[sheetname]</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,[Headers]</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,[FullFilePath]</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,ROW_NUMBER()OVER(ORDER BY (SELECT 1)) Rn</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,ROW_NUMBER()OVER(PARTITION BY [Headers] ORDER BY [Headers]) Grp</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,DENSE_RANK()OVER(ORDER BY [Headers]) Grp1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">FROM [dbo].[Tbl_PyImpExp1]</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">SELECT *,FIRST_VALUE([Filename]) OVER (PARTITION BY Grp1 ORDER BY Grp ASC) AS TableName</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">INTO Tbl_PyImpExp2 </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">FROM FileList </code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">END</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">IF EXISTS (SELECT 1 FROM Tbl_PyImpExp2)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">BEGIN</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">DECLARE @I INT = 1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@SQL NVARCHAR(MAX) =N'</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: blue;">SET @PythonScript = CONCAT('</code></div>
<div style="background-color: white;">
<code style="color: black;">import pandas </code><code style="color: #006699; font-weight: bold;">as</code> <code style="color: black;">pd</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">from</code> <code style="color: black;">revoscalepy import RxSqlServerData, rx_data_step</code></div>
<div style="background-color: white;">
<code style="color: black;">sqlConnString = </code><code style="color: blue;">"Driver=SQL Server;Server=Serv; ',@DBConnectionString,'"</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: black;">if ImportAll ==1 </code><code style="color: grey;">and</code> <code style="color: black;">CombineTarget==1:</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">FinalImport=pd.DataFrame()</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: #006699; font-weight: bold;">index</code><code style="color: black;">, row </code><code style="color: grey;">in</code> <code style="color: black;">InputDataSet.iterrows():</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">Tbl = </code><code style="color: blue;">""</code><code style="color: black;">.</code><code style="color: grey;">join</code><code style="color: black;">(T </code><code style="color: #006699; font-weight: bold;">for</code> <code style="color: black;">T </code><code style="color: grey;">in</code> <code style="color: black;">row[</code><code style="color: blue;">"TableName"</code><code style="color: black;">] if T.isalnum())</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">Import = pd.read_excel(row[</code><code style="color: blue;">"FullFilePath"</code><code style="color: black;">], sheetname=row[</code><code style="color: blue;">"sheetname"</code><code style="color: black;">], na_filter=</code><code style="color: #006699; font-weight: bold;">False</code><code style="color: black;">).astype(str)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">Import[</code><code style="color: blue;">"ImportKey"</code><code style="color: black;">] = row[</code><code style="color: blue;">"TableName"</code><code style="color: black;">]+</code><code style="color: blue;">"_"</code><code style="color: black;">+row[</code><code style="color: blue;">"sheetname"</code><code style="color: black;">]</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">FinalImport=FinalImport.append(Import,ignore_index=</code><code style="color: #006699; font-weight: bold;">True</code><code style="color: black;">)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">if </code><code style="color: grey;">not</code> <code style="color: black;">FinalImport.empty:</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: black;">sqlDS = RxSqlServerData(connection_string = sqlConnString,</code><code style="color: #006699; font-weight: bold;">table</code> <code style="color: black;">= Tbl)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: black;">rx_data_step(input_data = FinalImport, output_file = sqlDS,overwrite = </code><code style="color: #006699; font-weight: bold;">True</code><code style="color: black;">)</code><code style="color: blue;">')</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">WHILE @I <= (SELECT MAX(Grp1) FROM Tbl_PyImpExp2)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">SET @SQL = CONCAT('</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">FullFilePath,sheetname,TableName </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">Tbl_PyImpExp2 </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">Grp1 = </code><code style="color: blue;">',@I)</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">EXEC sp_execute_external_script</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">@language = N'</code><code style="color: black;">Python</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@script = @PythonScript</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@input_data_1 = @SQL</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@params = N'</code><code style="color: black;">@ImportAll </code><code style="color: #006699; font-weight: bold;">BIT</code><code style="color: black;">,@CombineTarget </code><code style="color: #006699; font-weight: bold;">BIT</code><code style="color: black;">,@Serv NVARCHAR(200)</code><code style="color: blue;">'</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@Serv = @Serv</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,@ImportAll = @ImportAll</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,@CombineTarget = @CombineTarget</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">SET @I = @I + 1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">END</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">DROP TABLE IF EXISTS Tbl_PyImpExp1,Tbl_PyImpExp2</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END ELSE PRINT '</code><code style="color: black;">Invalid parameters: If ImportAll = 0 </code><code style="color: #006699; font-weight: bold;">then</code> <code style="color: black;">pass Excel file & Sheet </code><code style="color: #006699; font-weight: bold;">Name</code> <code style="color: #006699; font-weight: bold;">as</code> <code style="color: black;">input. If ImportAll = 1 </code><code style="color: #006699; font-weight: bold;">then</code> <code style="color: black;">pass Excel file & Sheet </code><code style="color: #006699; font-weight: bold;">Name</code> <code style="color: black;">blank</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END ELSE PRINT '</code><code style="color: black;">Invalid folder path</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">END ELSE PRINT '</code><code style="color: black;">Import folder path </code><code style="color: grey;">or</code> <code style="color: #006699; font-weight: bold;">database</code> <code style="color: #006699; font-weight: bold;">connection</code> <code style="color: black;">string need </code><code style="color: #006699; font-weight: bold;">to</code> <code style="color: black;">be mentioned</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">END TRY</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">BEGIN CATCH</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: blue;">PRINT '</code><code style="color: black;">Issue while executing this SP, please </code><code style="color: #006699; font-weight: bold;">check</code> <code style="color: black;">whether there </code><code style="color: #006699; font-weight: bold;">is</code> <code style="color: black;">permission </code><code style="color: #006699; font-weight: bold;">to</code> <code style="color: #006699; font-weight: bold;">execute</code> <code style="color: black;">the script / </code><code style="color: #006699; font-weight: bold;">to</code> <code style="color: black;">access the folder </code><code style="color: grey;">and</code> <code style="color: black;">input params are valid'</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">END</code> <code style="color: black;">CATCH</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">END</code></div>
</div>
<br />
<br />
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></span>
<br />
<div style="background-color: white; border: 0px none rgb(42, 42, 42); font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Please see my <i><a href="https://social.technet.microsoft.com/wiki/contents/articles/53063.sql-server-importexport-to-excel-using-python-script.aspx" target="_blank">technet wiki article</a></i> for detailed explanation on this topic with examples and screenshots and you can also download the scripts and SSRS report from <i><a href="https://gallery.technet.microsoft.com/scriptcenter/SQL-Server-ImportExport-to-89e10784" target="_blank">technet gallery site</a></i>.</span></div>
<div style="background-color: white; border: 0px none rgb(42, 42, 42); font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="background-color: white; border: 0px none rgb(42, 42, 42); font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a; font-family: "arial" , "helvetica" , sans-serif; font-size: small;">We need to be careful with indentations in python script as formatting them without understanding the code can result in error.</span></div>
<div style="background-color: white; border: 0px none rgb(42, 42, 42); font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="background-color: white; border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br /></div>
<div style="background-color: white; border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<h3 style="margin: 0px; position: relative;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>See Also:</i></span></h3>
<br />
<ul style="line-height: 1.4; margin: 0.5em 0px; padding: 0px 2.5em;">
<li style="margin: 0px 0px 0.25em; padding: 0px;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://www.allaboutmssql.com/p/t-sql-scripts_19.html" style="color: #0b5394; text-decoration-line: none;" target="_blank">My other T-SQL posts</a></span></li>
</ul>
</div>
<br /></div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-13121050109159240232019-07-06T02:34:00.002+05:302019-07-06T02:41:48.184+05:30T-SQL : Data profiling in On-premise SQL Server / Azure SQL database<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how we can do data profiling on On-premise SQL Server / Azure SQL database tables using T-SQL script.</span></span><br />
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></span>
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-variant-east-asian: normal; font-variant-numeric: normal;">Data profiling provides below high level information about data:</span><br style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;" /><span style="font-variant-east-asian: normal; font-variant-numeric: normal;">
</span></span></span><br />
<ul style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px 0px 0px 40px; position: static;">
<li style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: black;">Number of rows and size of the data in the object, date for the most recent update of the data and the object schema</span>
</span></li>
<li style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: black;">Number of null records, distinct values for columns and data type of column</span>
</span></li>
<li style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Minimum, maximum values for numeric columns and check for existence of special characters, leading/trailing spaces for columns of string data type </span></li>
</ul>
<div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Two stored procedures are created to generate data profiling output. Inside the stored procedure, iteration on each table columns are performed using system catalog views to apply the MIN, MAX and other functions on each column to get the data profiling output<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">1) usp_DataProfiling<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="color: #2a2a2a; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">2) usp_DataProfiling_Metadata</span></div>
</div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below are the various input parameters of stored procedure's and it’s usage details:<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">1) usp_DataProfiling<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;">
</span><br style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;" /><span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;">
</span></span><br />
<table style="color: #2a2a2a; margin-left: 1px;">
<tbody style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> Parameter</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Description </span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@Report</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">1 = ColumnDataProfiling<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
2 = ColumnUniqueValues</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@SchemaName</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Schema name for which data profiling needs to be done. Multiple values can be passed separated by comma</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@ObjectlisttoSearch</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">List of tables separated by comma for which data profiling needs to be done </span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@ExcludeTables</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Tables to be excluded from data profiling output. Input format: Schemaname.tablename. Multiple values can be passed separated by comma<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@ExcludeColumns</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Columns to be excluded from data profiling output. Multiple values can be passed separated by comma</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@ExcludeDataType</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Data types to be excluded from data profiling output. Multiple values can be passed separated by comma. These data types are excluded by default 'geography','varbinary','binary','text', 'ntext', 'image', 'hierarchyid', 'xml', 'sql_variant' as they are not
really relevant for data profiling</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@RestrictCharlength<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Resultset shows only one record describing the details of the field, if the field has maximum character length greater than 100 else resultset will show one record per unique value in that field. This default value can be overwritten using this parameter<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">@RestrictNoOfUniqueValues<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Resultset shows only one record describing the details of the field, if that field has more than 50 unique values else resultset will show one record per unique value in that field. This default value can be overwritten using this parameter<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></td>
</tr>
</tbody>
</table>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;">
</span><br style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;" /><span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;">
</span></span><br />
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;"><span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></span>
<br />
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">2) usp_DataProfiling_Metadata<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;">
</span><br style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;" /><span style="font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px;">
</span></span><br />
<table style="color: #2a2a2a; font-size: 12.13px; margin-left: 1px;">
<tbody style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;"> Parameter</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Description </span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;"> @Report</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">1 = TableStats<br style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;" />
2 = TableColumnMetadata</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;"> @SchemaName</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Schema name for which data profiling needs to be done. Multiple values can be passed separated by comma</span></td>
</tr>
<tr style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px; position: static;">
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;"> @ObjectlisttoSearch</span></td>
<td style="border-bottom-color: rgb(42, 42, 42); border-bottom-style: none; border-bottom-width: 0px; border-image-outset: 0; border-image-repeat: stretch; border-image-slice: 100%; border-image-source: none; border-image-width: 1; border-left-color: rgb(42, 42, 42); border-left-style: none; border-left-width: 0px; border-right-color: rgb(42, 42, 42); border-right-style: none; border-right-width: 0px; border-top-color: rgb(42, 42, 42); border-top-style: none; border-top-width: 0px; line-height: 16.98px; max-width: none; overflow: visible; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px; position: static;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">List of tables separated by comma for which data profiling needs to be done</span> </td></tr>
</tbody></table>
</div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br /></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br /></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a; font-family: "arial" , "helvetica" , sans-serif;">Data Profiling using T-SQL script:</span><br />
<span style="color: #2a2a2a; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<h2 style="text-align: left;">
<span style="color: #2a2a2a; font-family: "arial" , "helvetica" , sans-serif;"><b>1) usp_DataProfiling</b></span></h2>
</div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: grey;">OR</code> <code style="color: #006699; font-weight: bold;">ALTER</code> <code style="color: #006699; font-weight: bold;">PROCEDURE</code> <code>usp_DataProfiling</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@Report TINYINT , </code><code style="color: #008200;">--1 = 'ColumnDataProfiling', 2 = 'ColumnUniqueValues'</code></div>
<div style="background-color: white;">
<code> </code><code>@SchemaName NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code><code>,</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@ObjectlisttoSearch NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),</code></div>
<div style="background-color: white;">
<code> </code><code>@ExcludeTables NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code><code>,</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@ExcludeColumns NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code><code>,</code></div>
<div style="background-color: white;">
<code> </code><code>@ExcludeDataType NVARCHAR(100) = N</code><code style="color: blue;">''</code><code>,</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@RestrictCharlength </code><code style="color: #006699; font-weight: bold;">INT</code><code>,</code></div>
<div style="background-color: white;">
<code> </code><code>@RestrictNoOfUniqueValues </code><code style="color: #006699; font-weight: bold;">INT</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">AS</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SET</code> <code>NOCOUNT </code><code style="color: #006699; font-weight: bold;">ON</code><code>;</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SET</code> <code>ANSI_WARNINGS </code><code style="color: #006699; font-weight: bold;">OFF</code><code>;</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SET</code> <code>ANSI_NULLS </code><code style="color: #006699; font-weight: bold;">ON</code><code>;</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>@RestrictCharlength = IIF(@RestrictCharlength </code><code style="color: #006699; font-weight: bold;">IS</code> <code style="color: grey;">NULL</code> <code style="color: grey;">OR</code> <code>@RestrictCharlength = </code><code style="color: blue;">''</code><code>,100,@RestrictCharlength)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>@RestrictNoOfUniqueValues = IIF(@RestrictNoOfUniqueValues </code><code style="color: #006699; font-weight: bold;">IS</code> <code style="color: grey;">NULL</code> <code style="color: grey;">OR</code> <code>@RestrictNoOfUniqueValues = </code><code style="color: blue;">''</code><code>,50,@RestrictNoOfUniqueValues)</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">DECLARE</code> <code>@TableColList </code><code style="color: #006699; font-weight: bold;">TABLE</code> <code>(Id </code><code style="color: #006699; font-weight: bold;">INT</code> <code>IDENTITY(1,1),Tbl NVARCHAR(128),colname NVARCHAR(200),ColType NVARCHAR(150))</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code>IF </code><code style="color: deeppink;">ISNULL</code><code>(@SchemaName,</code><code style="color: blue;">''</code><code>) <> </code><code style="color: blue;">''</code> <code style="color: grey;">OR</code> <code style="color: deeppink;">ISNULL</code><code>(@ObjectlisttoSearch,</code><code style="color: blue;">''</code><code>) <> </code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code>@TableColList</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">DISTINCT</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) TableName</code></div>
<div style="background-color: white;">
<code> </code><code>,C.</code><code style="color: #006699; font-weight: bold;">name</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>TY.is_user_defined = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">name</code> <code style="color: #006699; font-weight: bold;">FROM</code> <code>sys.types </code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code>system_type_id = user_type_id</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>system_type_id = TY.system_type_id)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ELSE</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>Sys.tables T</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: grey;">JOIN</code> <code>sys.columns C</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>T.object_id = C.object_id</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: grey;">JOIN</code> <code>sys.types TY </code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>C.[user_type_id] = TY.[user_type_id]</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #008200;">-- Ignore the datatypes that are not required </code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">WHERE</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code> <code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: blue;">'geography'</code><code>,</code><code style="color: blue;">'varbinary'</code><code>,</code><code style="color: blue;">'binary'</code><code>,</code><code style="color: blue;">'text'</code><code>, </code><code style="color: blue;">'ntext'</code><code>, </code><code style="color: blue;">'image'</code><code>, </code><code style="color: blue;">'hierarchyid'</code><code>, </code><code style="color: blue;">'xml'</code><code>, </code><code style="color: blue;">'sql_variant'</code><code>) </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(Schema_name(T.schema_id) </code><code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@SchemaName, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">OR</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) </code><code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ObjectlisttoSearch, </code><code style="color: blue;">','</code><code>)))</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(TY.</code><code style="color: #006699; font-weight: bold;">name</code> <code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ExcludeDataType, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code> <code>= TY.</code><code style="color: #006699; font-weight: bold;">name</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(C.</code><code style="color: #006699; font-weight: bold;">name</code> <code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ExcludeColumns, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>C.</code><code style="color: #006699; font-weight: bold;">name</code> <code>= C.</code><code style="color: #006699; font-weight: bold;">name</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) </code><code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ExcludeTables, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) = CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>))</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">ELSE</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">INSERT</code> <code>@TableColList</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">DISTINCT</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) TableName</code></div>
<div style="background-color: white;">
<code> </code><code>,C.</code><code style="color: #006699; font-weight: bold;">name</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>TY.is_user_defined = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">name</code> <code style="color: #006699; font-weight: bold;">FROM</code> <code>sys.types </code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code>system_type_id = user_type_id</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>system_type_id = TY.system_type_id)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ELSE</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>Sys.tables T</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: grey;">JOIN</code> <code>sys.columns C</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>T.object_id = C.object_id</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: grey;">JOIN</code> <code>sys.types TY </code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>C.[user_type_id] = TY.[user_type_id]</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #008200;">-- Ignore the datatypes that are not required </code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">WHERE</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code> <code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: blue;">'geography'</code><code>,</code><code style="color: blue;">'varbinary'</code><code>,</code><code style="color: blue;">'binary'</code><code>,</code><code style="color: blue;">'text'</code><code>, </code><code style="color: blue;">'ntext'</code><code>, </code><code style="color: blue;">'image'</code><code>, </code><code style="color: blue;">'hierarchyid'</code><code>, </code><code style="color: blue;">'xml'</code><code>, </code><code style="color: blue;">'sql_variant'</code><code>) </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(TY.</code><code style="color: #006699; font-weight: bold;">name</code> <code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ExcludeDataType, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code> <code>= TY.</code><code style="color: #006699; font-weight: bold;">name</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(C.</code><code style="color: #006699; font-weight: bold;">name</code> <code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ExcludeColumns, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>C.</code><code style="color: #006699; font-weight: bold;">name</code> <code>= C.</code><code style="color: #006699; font-weight: bold;">name</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">AND</code> <code>(CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) </code><code style="color: grey;">NOT</code> <code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ExcludeTables, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) = CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>))</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">DROP</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>IF EXISTS #Final</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>#Final (Id </code><code style="color: #006699; font-weight: bold;">BIGINT</code> <code>IDENTITY(1,1),TableName NVARCHAR(128),ColumnName NVARCHAR(200),ColumnType NVARCHAR(150),ColumnUniqueValues NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),UniqueValueOccurance </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MissingDataRowCount </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MinValue NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),MaxValue NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),SpecialCharacters </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,LeadingTrailingSpaces </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MinFieldValueLen </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MaxFieldValueLen </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,Comment NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>))</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">DROP</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>IF EXISTS #</code><code style="color: #006699; font-weight: bold;">temp</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>#</code><code style="color: #006699; font-weight: bold;">temp</code> <code>(Id </code><code style="color: #006699; font-weight: bold;">BIGINT</code> <code>IDENTITY(1,1),TableName NVARCHAR(128),ColumnName NVARCHAR(200),Cnt </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MaxLen </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MinLen </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MissingDataCount </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,MinValue NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),MaxValue NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>),SpecialCharacters </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>,LeadingTrailingSpaces </code><code style="color: #006699; font-weight: bold;">BIGINT</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">DECLARE</code> <code>@I </code><code style="color: #006699; font-weight: bold;">INT</code> <code>= 1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@SQL NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code> </code><code>,@tblname NVARCHAR(128)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@Colname NVARCHAR(200)</code></div>
<div style="background-color: white;">
<code> </code><code>,@ColType NVARCHAR(150)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@Cnt </code><code style="color: #006699; font-weight: bold;">BIGINT</code></div>
<div style="background-color: white;">
<code> </code><code>,@MaxLen </code><code style="color: #006699; font-weight: bold;">BIGINT</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@MinLen </code><code style="color: #006699; font-weight: bold;">BIGINT</code></div>
<div style="background-color: white;">
<code> </code><code>,@MissingData </code><code style="color: #006699; font-weight: bold;">BIGINT</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@MaxVal NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code> </code><code>,@MinVal NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@MinMAxSQL NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code> </code><code>,@SpecialCharacters </code><code style="color: #006699; font-weight: bold;">BIGINT</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@LeadingTrailingSpaces </code><code style="color: #006699; font-weight: bold;">BIGINT</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code>WHILE @I <= (</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">MAX</code><code>(Id) </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>@TableColList)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>@Colname = QUOTENAME(colname),@tblname = Tbl,@ColType = ColType </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>@TableColList</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code>Id = @I</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code>@MinMAxSQL = </code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>@ColType </code><code style="color: grey;">IN</code> <code>(</code><code style="color: blue;">'date'</code><code>,</code><code style="color: blue;">'datetime'</code><code>,</code><code style="color: blue;">'datetime2'</code><code>,</code><code style="color: blue;">'datetimeoffset'</code><code>,</code><code style="color: blue;">'time'</code><code>,</code><code style="color: blue;">'timestamp'</code><code>)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">THEN</code> <code>CONCAT(</code><code style="color: blue;">' FORMAT (MIN('</code><code>,@Colname,</code><code style="color: blue;">'), '</code><code style="color: blue;">'yyyy-MM-dd,hh:mm:ss'</code><code style="color: blue;">') MinValue,FORMAT (MAX('</code><code>,@Colname,</code><code style="color: blue;">'), '</code><code style="color: blue;">'yyyy-MM-dd,hh:mm:ss'</code><code style="color: blue;">') MAXValue'</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">WHEN</code> <code>@ColType = </code><code style="color: blue;">'bit'</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'0 AS MinValue,1 AS MaxValue'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">ELSE</code> <code>CONCAT(</code><code style="color: blue;">'CASE WHEN EXISTS (SELECT 1 FROM '</code><code>,@tblname,</code><code style="color: blue;">' WHERE ISNUMERIC('</code><code>,@Colname,</code><code style="color: blue;">') = 0)'</code><code>,</code><code style="color: blue;">'THEN NULL ELSE MIN('</code><code>,@Colname,</code><code style="color: blue;">') END MinValue</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,CASE WHEN EXISTS (SELECT 1 FROM '</code><code>,@tblname,</code><code style="color: blue;">' WHERE ISNUMERIC('</code><code>,@Colname,</code><code style="color: blue;">') = 0)'</code><code>,</code><code style="color: blue;">'THEN NULL ELSE MAX('</code><code>,@Colname,</code><code style="color: blue;">') END MAXValue'</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">EXEC</code> <code>(</code><code style="color: blue;">';WITH CTE AS (</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">SELECT COUNT_BIG(DISTINCT '</code><code>+@Colname+</code><code style="color: blue;">') Cnt</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,MAX(LEN('</code><code>+@Colname+</code><code style="color: blue;">')) MaxLen</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,MIN(LEN('</code><code>+@Colname+</code><code style="color: blue;">')) MinLen</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,SUM(CASE WHEN '</code><code>+@Colname+</code><code style="color: blue;">' IS NULL OR CAST('</code><code>+@Colname+</code><code style="color: blue;">' AS VARCHAR(MAX)) = '</code><code style="color: blue;">''</code><code style="color: blue;">' THEN 1 ELSE 0 END) MissingData</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,'</code><code>+@MinMAxSQL+</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">,CASE WHEN '</code><code style="color: blue;">''</code><code>+@ColType+</code><code style="color: blue;">''</code><code style="color: blue;">' IN ('</code><code style="color: blue;">'nvarchar'</code><code style="color: blue;">','</code><code style="color: blue;">'varchar'</code><code style="color: blue;">','</code><code style="color: blue;">'nchar'</code><code style="color: blue;">','</code><code style="color: blue;">'char'</code><code style="color: blue;">') </code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">THEN SUM(CASE WHEN '</code><code>+@Colname+</code><code style="color: blue;">' LIKE '</code><code style="color: blue;">'%[^a-zA-Z0-9 ]%'</code><code style="color: blue;">' THEN 1 ELSE 0 END) </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">ELSE NULL END SpecialCharacters</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,CASE WHEN '</code><code style="color: blue;">''</code><code>+@ColType+</code><code style="color: blue;">''</code><code style="color: blue;">' IN ('</code><code style="color: blue;">'nvarchar'</code><code style="color: blue;">','</code><code style="color: blue;">'varchar'</code><code style="color: blue;">','</code><code style="color: blue;">'nchar'</code><code style="color: blue;">','</code><code style="color: blue;">'char'</code><code style="color: blue;">') </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">THEN SUM(CASE WHEN ISNULL(DATALENGTH('</code><code>+@Colname+</code><code style="color: blue;">'),'</code><code style="color: blue;">''</code><code style="color: blue;">') = ISNULL(DATALENGTH(RTRIM(LTRIM('</code><code>+@Colname+</code><code style="color: blue;">'))),'</code><code style="color: blue;">''</code><code style="color: blue;">') THEN 0 ELSE 1 END) </code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">ELSE NULL END LeadingTrailingSpaces</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">FROM '</code><code>+@tblname+</code><code style="color: blue;">' )</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">INSERT #temp(TableName,ColumnName,Cnt,MaxLen,MinLen,MissingDataCount,MinValue,MaxValue,SpecialCharacters,LeadingTrailingSpaces)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: blue;">SELECT '</code><code style="color: blue;">''</code><code>+@tblname+</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code>+@Colname+</code><code style="color: blue;">''</code><code style="color: blue;">',Cnt,ISNULL(MaxLen,0) MaxLen,ISNULL(MinLen,0) MinLen,ISNULL(MissingData,0) MissingData,MinValue,MAXValue</code></div>
<div style="background-color: white;">
<code> </code><code style="color: blue;">,ISNULL(SpecialCharacters,0) SpecialCharacters,ISNULL(LeadingTrailingSpaces,0) LeadingTrailingSpaces FROM CTE'</code><code>)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>@Cnt = Cnt,@MaxLen = MaxLen,@MinLen = MinLen,@MissingData = MissingDataCount,@MinVal=MinValue,@MaxVal=MAXValue</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@SpecialCharacters = SpecialCharacters ,@LeadingTrailingSpaces = LeadingTrailingSpaces </code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>#</code><code style="color: #006699; font-weight: bold;">temp</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code>Id = @I </code><code style="color: grey;">AND</code> <code>TableName = @tblname </code><code style="color: grey;">AND</code> <code>ColumnName = @Colname</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code>IF </code><code style="color: deeppink;">ISNULL</code><code>(@MaxLen,</code><code style="color: blue;">''</code><code>) < @RestrictCharlength </code><code style="color: grey;">AND</code> <code style="color: deeppink;">ISNULL</code><code>(@Cnt,</code><code style="color: blue;">''</code><code>) < @RestrictNoOfUniqueValues </code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SET</code> <code>@SQL = CONCAT(</code><code style="color: blue;">'SELECT '</code><code style="color: blue;">''</code><code>,@tblname,</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code>,@Colname,</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code>,@ColType,</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code>,@Colname,</code><code style="color: blue;">',COUNT_BIG('</code><code>,@Colname,</code><code style="color: blue;">'),'</code><code>,@MissingData,</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code>,@MinVal,</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code>,@MaxVal,</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code>,@SpecialCharacters,</code><code style="color: blue;">','</code><code>,@LeadingTrailingSpaces,</code><code style="color: blue;">','</code><code>,@MinLen,</code><code style="color: blue;">','</code><code>,@MaxLen,</code><code style="color: blue;">','</code><code>,</code><code style="color: blue;">''</code><code style="color: blue;">''</code><code>,</code><code style="color: blue;">'This field has Unique values = '</code><code>,@Cnt,</code><code style="color: blue;">''</code><code style="color: blue;">''</code><code>,</code><code style="color: blue;">' FROM '</code><code>,@tblname,</code><code style="color: blue;">' GROUP BY '</code><code>,@Colname)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">INSERT</code> <code>#Final (TableName,ColumnName,ColumnType,ColumnUniqueValues,UniqueValueOccurance,MissingDataRowCount,MinValue,MaxValue,SpecialCharacters,LeadingTrailingSpaces,MinFieldValueLen,MaxFieldValueLen,Comment)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">EXEC</code><code>(@SQL)</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ELSE</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">INSERT</code> <code>#Final (TableName,ColumnName,ColumnType,MissingDataRowCount,MinValue,MaxValue,SpecialCharacters,LeadingTrailingSpaces,MinFieldValueLen,MaxFieldValueLen,Comment)</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>@tblname,@Colname,@ColType,@MissingData,@MinVal,@MaxVal,@SpecialCharacters,@LeadingTrailingSpaces,@MinLen,@MaxLen,CONCAT(</code><code style="color: blue;">'This field has Unique values = '</code><code>,@Cnt)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SET</code> <code>@I = @I + 1</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code>IF @Report = 1</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">DISTINCT</code> <code>TableName,ColumnName,ColumnType,MissingDataRowCount,MinValue,MaxValue,SpecialCharacters</code></div>
<div style="background-color: white;">
<code> </code><code>,LeadingTrailingSpaces,MinFieldValueLen,MaxFieldValueLen,Comment</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>#Final</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ORDER</code> <code style="color: #006699; font-weight: bold;">BY</code> <code>TableName,ColumnName</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: #f8f8f8;">
<code> </code> </div>
<div style="background-color: white;">
<code> </code><code>IF @Report = 2</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>TableName,ColumnName,ColumnUniqueValues,UniqueValueOccurance,Comment</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>#Final</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">ORDER</code> <code style="color: #006699; font-weight: bold;">BY</code> <code>TableName,ColumnName</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
</div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span>
<br />
<h2 style="text-align: left;">
<span style="color: #2a2a2a;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>2) usp_DataProfiling_Metadata</b></span></span></h2>
</div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: grey;">OR</code> <code style="color: #006699; font-weight: bold;">ALTER</code> <code style="color: #006699; font-weight: bold;">PROCEDURE</code> <code>usp_DataProfiling_Metadata</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@Report TINYINT , </code><code style="color: #008200;">--1 = 'TableStats', 2 = 'TableColumnMetadata'</code></div>
<div style="background-color: white;">
<code> </code><code>@SchemaName NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code><code>,</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>@ObjectlisttoSearch NVARCHAR(</code><code style="color: #006699; font-weight: bold;">MAX</code><code>) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">AS</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SET</code> <code>NOCOUNT </code><code style="color: #006699; font-weight: bold;">ON</code><code>;</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">DROP</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>IF EXISTS #TblList</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>#TblList(Id </code><code style="color: #006699; font-weight: bold;">INT</code> <code>IDENTITY(1,1),TableName NVARCHAR(200) )</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">DROP</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>IF EXISTS #Tblstats</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">CREATE</code> <code style="color: #006699; font-weight: bold;">TABLE</code> <code>#Tblstats (TableName NVARCHAR(200),NoOfRows NVARCHAR(100),ReservedSpace NVARCHAR(100)</code></div>
<div style="background-color: white;">
<code> </code><code>,DataSpace NVARCHAR(100),IndexSize NVARCHAR(100),UnusedSpace NVARCHAR(100)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,LastUserUpdate DATETIME)</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code>IF </code><code style="color: deeppink;">ISNULL</code><code>(@SchemaName,</code><code style="color: blue;">''</code><code>) <> </code><code style="color: blue;">''</code> <code style="color: grey;">OR</code> <code style="color: deeppink;">ISNULL</code><code>(@ObjectlisttoSearch,</code><code style="color: blue;">''</code><code>) <> </code><code style="color: blue;">''</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code>#TblList (TableName)</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code>CONCAT(SCHEMA_NAME(SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,</code><code style="color: #006699; font-weight: bold;">name</code><code>) TableName </code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>Sys.tables</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">WHERE</code> <code>(Schema_name(schema_id) </code><code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@SchemaName, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">OR</code> <code>CONCAT(SCHEMA_NAME(SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,</code><code style="color: #006699; font-weight: bold;">name</code><code>) </code><code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ObjectlisttoSearch, </code><code style="color: blue;">','</code><code>)))</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">ELSE</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code>#TblList (TableName)</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code>CONCAT(SCHEMA_NAME(SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,</code><code style="color: #006699; font-weight: bold;">name</code><code>) TableName </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>Sys.tables</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">DECLARE</code> <code>@Tblstats </code><code style="color: #006699; font-weight: bold;">TABLE</code><code>(TableName NVARCHAR(200),NoOfRows NVARCHAR(100),ReservedSpace NVARCHAR(100)</code></div>
<div style="background-color: white;">
<code> </code><code>,DataSpace NVARCHAR(100),IndexSize NVARCHAR(100),UnusedSpace NVARCHAR(100)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>)</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">DECLARE</code> <code>@I </code><code style="color: #006699; font-weight: bold;">INT</code> <code>= 1</code></div>
<div style="background-color: white;">
<code> </code><code>,@tblname NVARCHAR(128) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,@last_user_update DATETIME</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code>WHILE @I <= (</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: deeppink;">COUNT</code><code>(1) </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>#TblList)</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code>@tblname=TableName </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>#TblList </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code>Id = @I</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code>@Tblstats </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">EXEC</code> <code>sp_spaceused @tblname; </code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">TOP</code> <code>1 @last_user_update=last_user_update </code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>sys.dm_db_index_usage_stats </code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">WHERE</code> <code>object_id = OBJECT_ID(@tblname)</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">ORDER</code> <code style="color: #006699; font-weight: bold;">BY</code> <code>last_user_update </code><code style="color: #006699; font-weight: bold;">DESC</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code>#Tblstats(TableName,NoOfRows,ReservedSpace,DataSpace,IndexSize,UnusedSpace,LastUserUpdate)</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code>@tblname,NoOfRows,ReservedSpace,DataSpace,IndexSize,UnusedSpace,@last_user_update </code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>@Tblstats</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">DELETE</code> <code style="color: #006699; font-weight: bold;">FROM</code> <code>@Tblstats</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">SET</code> <code>@I = @I + 1</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code>IF @Report = 1</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code>;</code><code style="color: #006699; font-weight: bold;">WITH</code> <code>Systbl</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">AS</code></div>
<div style="background-color: white;">
<code> </code><code>(</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">DISTINCT</code> <code>CONCAT(SCHEMA_NAME(SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,</code><code style="color: #006699; font-weight: bold;">name</code><code>) TableName </code></div>
<div style="background-color: white;">
<code> </code><code>,modify_date TableSchema_LastModifyDate</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>is_replicated = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsReplicated</code></div>
<div style="background-color: white;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>is_filetable = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsFileTable</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>is_memory_optimized = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsMemoryOptimized</code></div>
<div style="background-color: white;">
<code> </code><code>,temporal_type_desc TemporalTypeDesc</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>is_remote_data_archive_enabled = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsStretchEnabled</code></div>
<div style="background-color: white;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>is_external = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsExternal</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>is_node = 1 </code><code style="color: grey;">OR</code> <code>is_edge = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code>IsGraphTable</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>sys.tables ST</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">JOIN</code> <code>#TblList T</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>CONCAT(SCHEMA_NAME(SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,</code><code style="color: #006699; font-weight: bold;">name</code><code>) </code><code style="color: #006699; font-weight: bold;">COLLATE</code> <code>DATABASE_DEFAULT = T.TableName </code><code style="color: #006699; font-weight: bold;">COLLATE</code> <code>DATABASE_DEFAULT</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>)</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code>B.*,A.TableSchema_LastModifyDate </code></div>
<div style="background-color: #f8f8f8;">
<code>,A.IsMemoryOptimized</code></div>
<div style="background-color: white;">
<code>,A.IsExternal</code></div>
<div style="background-color: #f8f8f8;">
<code>,A.IsStretchEnabled</code></div>
<div style="background-color: white;">
<code>,A.IsFileTable</code></div>
<div style="background-color: #f8f8f8;">
<code>,A.IsGraphTable</code></div>
<div style="background-color: white;">
<code>,A.IsReplicated</code></div>
<div style="background-color: #f8f8f8;">
<code>,A.TemporalTypeDesc</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>Systbl A</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: grey;">JOIN</code> <code>#Tblstats B</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">ON</code> <code>A.TableName </code><code style="color: #006699; font-weight: bold;">COLLATE</code> <code>DATABASE_DEFAULT = B.TableName </code><code style="color: #006699; font-weight: bold;">COLLATE</code> <code>DATABASE_DEFAULT</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
<code> </code> </div>
<div style="background-color: #f8f8f8;">
<code> </code><code>IF @Report = 2</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">BEGIN</code></div>
<div style="background-color: #f8f8f8;">
</div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">DISTINCT</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) TableName</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,C.</code><code style="color: #006699; font-weight: bold;">name</code> <code>ColumnName</code></div>
<div style="background-color: white;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>TY.is_user_defined = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">name</code> <code style="color: #006699; font-weight: bold;">FROM</code> <code>sys.types </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code>system_type_id = user_type_id</code></div>
<div style="background-color: white;">
<code> </code><code style="color: grey;">AND</code> <code>system_type_id = TY.system_type_id)</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">ELSE</code> <code>TY.</code><code style="color: #006699; font-weight: bold;">name</code></div>
<div style="background-color: white;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>DataType</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,C.max_length</code></div>
<div style="background-color: white;">
<code> </code><code>,C.</code><code style="color: #006699; font-weight: bold;">precision</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,C.scale</code></div>
<div style="background-color: white;">
<code> </code><code>,C.collation_name</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_nullable = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsNullable</code></div>
<div style="background-color: white;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_identity = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsIdentity</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_masked = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsMasked</code></div>
<div style="background-color: white;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_hidden = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsHidden</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_computed = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsComputed</code></div>
<div style="background-color: white;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_filestream = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsFileStream</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code>,</code><code style="color: deeppink;">CASE</code> <code style="color: #006699; font-weight: bold;">WHEN</code> <code>C.is_sparse = 1 </code><code style="color: #006699; font-weight: bold;">THEN</code> <code style="color: blue;">'Yes'</code> <code style="color: #006699; font-weight: bold;">ELSE</code> <code style="color: blue;">'No'</code> <code style="color: #006699; font-weight: bold;">END</code> <code style="color: #006699; font-weight: bold;">AS</code> <code>IsSparse</code></div>
<div style="background-color: white;">
<code> </code><code>,C.encryption_type_desc EncryptionTypeDesc</code></div>
<div style="background-color: #f8f8f8;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code>Sys.tables T</code></div>
<div style="background-color: white;">
<code style="color: grey;">JOIN</code> <code>sys.columns C</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>T.object_id = C.object_id</code></div>
<div style="background-color: white;">
<code style="color: grey;">JOIN</code> <code>sys.types TY </code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code>C.[user_type_id] = TY.[user_type_id]</code></div>
<div style="background-color: white;">
<code style="color: #006699; font-weight: bold;">WHERE</code> <code>(Schema_name(T.schema_id) </code><code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@SchemaName, </code><code style="color: blue;">','</code><code>))</code></div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: grey;">OR</code> <code>CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code>,T.</code><code style="color: #006699; font-weight: bold;">name</code><code>) </code><code style="color: grey;">IN</code> <code>(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code>value </code><code style="color: #006699; font-weight: bold;">FROM</code> <code>STRING_SPLIT(@ObjectlisttoSearch, </code><code style="color: blue;">','</code><code>)))</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
<div style="background-color: white;">
</div>
<div style="background-color: #f8f8f8;">
<code> </code><code style="color: #006699; font-weight: bold;">END</code></div>
</div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="color: #2a2a2a;"><span style="font-size: 12.13px;"><br /></span></span></div>
<div style="border: 0px none rgb(42, 42, 42); font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Please see my <a href="https://social.technet.microsoft.com/wiki/contents/articles/52865.t-sql-data-profiling-in-on-premise-sql-server-azure-sql-database.aspx" target="_blank"><i>technet wiki article</i></a> for detailed explanation on this topic with examples and screenshots and you can also download the scripts and SSRS report from <a href="https://gallery.technet.microsoft.com/scriptcenter/T-SQL-Data-profiling-in-On-8fa18dde" target="_blank"><i>technet gallery site</i></a>.</span></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br /></div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><i>See Also:</i></span></h3>
<br />
<ul style="text-align: left;">
<li><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://www.allaboutmssql.com/p/t-sql-scripts_19.html" target="_blank">My other T-SQL posts</a></span></li>
</ul>
</div>
<div style="border: 0px none rgb(42, 42, 42); color: #2a2a2a; font-size: 12.13px; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: 16.98px; max-width: none; overflow: visible; padding: 0px; position: static;">
<br /></div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-48033500003072190722019-05-04T02:17:00.001+05:302019-05-04T02:34:31.101+05:30T-SQL : Search for string or phrase in SQL Server database<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
In this post, let us see how to search for a string / phrase in SQL Server database using hybrid solution of T-SQL LIKE operator & R grep function. Currently the options that exists in SQL Server to perform a search operation are </div>
<ol style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<li>LIKE operator </li>
<li>Using string functions CHARINDEX, PATINDEX </li>
<li>Full text search </li>
</ol>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Consider below example: To search and return only records with string "VAT" . Expected result is to return record 1,5 & 6.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">DECLARE</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">TABLE</code> <code style="color: black;">(Id </code><code style="color: #006699; font-weight: bold;">INT</code><code style="color: black;">, Descrip </code><code style="color: #006699; font-weight: bold;">VARCHAR</code><code style="color: black;">(500))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">1,</code><code style="color: blue;">'my VAT calculation is incorrect'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">2,</code><code style="color: blue;">'Private number'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">3,</code><code style="color: blue;">'Innnovation model'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">4,</code><code style="color: blue;">'ELEVATE'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">5,</code><code style="color: blue;">'total VAT'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">INSERT</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">6,</code><code style="color: blue;">'VAT'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">* </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">Descrip </code><code style="color: grey;">LIKE</code> <code style="color: blue;">'VAT'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">* </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">Descrip </code><code style="color: grey;">LIKE</code> <code style="color: blue;">'%VAT'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">* </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">Descrip </code><code style="color: grey;">LIKE</code> <code style="color: blue;">'%VAT%'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">* </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">Descrip </code><code style="color: grey;">LIKE</code> <code style="color: blue;">'% VAT %'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">* </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">@Tmp </code><code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">Descrip </code><code style="color: grey;">LIKE</code> <code style="color: blue;">'% VAT'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjH4K4EWpw-dl2OXmQY99SlyR4YIlRT0bLRzg5xBek99EZ66BL8bMd_JnTRhy8vpDUUTpC5dKiWwqF49jxZF9T1Kbiy3Kdegp0ApCDfgOXuJnXC7h3ZCckI18AyJpRTmAMdBHjptbjkCOZ_/s1600/SS.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="594" data-original-width="529" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjH4K4EWpw-dl2OXmQY99SlyR4YIlRT0bLRzg5xBek99EZ66BL8bMd_JnTRhy8vpDUUTpC5dKiWwqF49jxZF9T1Kbiy3Kdegp0ApCDfgOXuJnXC7h3ZCckI18AyJpRTmAMdBHjptbjkCOZ_/s640/SS.JPG" width="568" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
As shown in above example, to do an exact search on string, there is no straight forward option using first two options mentioned above. However though it is possible with third option using Full text CONTAINS predicate. Full text catalog, unique index & full text index has to be created on the table on which search operation needs to be performed.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
If the exact search of string needs to be performed on the entire database then creating full text catalog, unique index & full text index on each and every table won’t be a viable option.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
With the hybrid approach <code>[</code>T-SQL LIKE operator & R grep function], let us see various search types that can be performed <code>[</code>Pattern Search, Exact Search, Multi pattern search and other search scenario's – based on collation, case sensitive/insensitive search and complex wildcard search].</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
</div>
<a name='more'></a><br />
<br />
<h1 style="background-color: white; clear: both; color: #707070; font-family: "Segoe UI Light", "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 36.3312px; font-weight: 100; line-height: 1.3;">
Pre-requisites</h1>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
We have used <a href="https://www.microsoft.com/en-us/sql-server/sql-server-2019" style="color: #00749e; cursor: pointer; outline: none; text-decoration-line: none;" target="_blank">SQL Server 2019 evaluation edition <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: auto !important; max-width: 100%; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a>on Windows 10 64 bit and <a href="https://cloudblogs.microsoft.com/sqlserver/2016/06/09/wideworldimporters-the-new-sql-server-sample-database/" style="color: #00749e; cursor: pointer; outline: none; text-decoration-line: none;" target="_blank">WideWorldImporters <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: auto !important; max-width: 100%; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a>SQL Server sample database for this example. In this example, we have made use of R services installed as part of SQL Server. <a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/install/sql-r-services-windows-install?view=sql-server-2017" style="color: #00749e; cursor: pointer; outline: none; text-decoration-line: none;" target="_blank">Install R services <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: auto !important; max-width: 100%; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a>and then from SSMS enable the external scripting feature. Restart the database engine and then verify the installation as mentioned in MSDN.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Below script / this approach will work starting from SQL Server 2016 and above (as execution of R language using T-SQL was introduced in SQL Server 2016). Also please note, no additional R packages need to be installed for this approach.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h1 style="background-color: white; clear: both; color: #707070; font-family: "Segoe UI Light", "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 36.3312px; font-weight: 100; line-height: 1.3;">
Explanation on how this approach works</h1>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
A stored procedure named "usp_SearchString" has been created. This stored procedure has the capability to do normal T-SQL LIKE operations as well as can search string using R grep function and this can be controlled through input parameter.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Output of the search operation will be stored in a table named "Tbl_SearchString". Also output will be displayed at the end of stored procedure execution.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Below are the various input parameters of stored procedure and it’s usage details:</div>
<table style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px; margin-left: 1px; width: 640px;"><tbody>
<tr><td>Parameter</td><td>Description</td></tr>
<tr><td>@SearchString</td><td>String to be searched</td></tr>
<tr><td>@SearchType</td><td>ES - Exact Search using R<br />
PS - Pattern Search using R<br />
MPS - Multi Pattern Search (OR condition) using R<br />
NTLS - Normal T-SQL Like Search</td></tr>
<tr><td>@Match</td><td>0 = LIKE Search,<br />
1 = NOT LIKE Search</td></tr>
<tr><td>@IgnoreCase</td><td>1 = case insensitive search,<br />
0 = Case sensitive search<br />
(If @IgnoreCase IS NULL then default : case insensitive search)</td></tr>
<tr><td>@SearchSQLMetadata</td><td>Search sql definitions for presence of input string.<br />
1 = Search,<br />
0 = Don't Search</td></tr>
<tr><td>@SchemaName</td><td>List of objects to be searched that fall under schema (Multiple schema's can be passed, separated by Comma)</td></tr>
<tr><td>@ObjectlisttoSearch</td><td> List of objects to be searched (Multiple tables can be passed, separated by Comma)</td></tr>
<tr><td>@SearchCollate</td><td>For @SearchType = NTLS if @IgnoreCase = 0.<br />
To search based on particular collation, default - COLLATE Latin1_General_CS_AS</td></tr>
</tbody></table>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
If both @ObjectlisttoSearch & @SchemaName are blank then entire database is searched including SQL object definitions</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<strong>Please note:</strong></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<strong></strong></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
@SearchType = ES, PS, MPS uses R script to search string</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
@SearchType = NTLS does the normal T-SQL LIKE search<br />
<br />
Multi string search should always be delimited by pipe. </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
@ObjectlisttoSearch, @SchemaName should always be delimited by comma if multiple values specified.</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h1 style="background-color: white; clear: both; color: #707070; font-family: "Segoe UI Light", "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 36.3312px; font-weight: 100; line-height: 1.3;">
<a href="https://www.blogger.com/null" name="Script" style="color: #00749e; cursor: pointer;"></a>Script</h1>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">USE <code>[</code>WideWorldImporters]</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">GO</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">--Note : Before compiling this SP, search for sqlConnString and provide Databasename, username & password for R SQL connection</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">CREATE OR ALTER PROC usp_SearchString ( @SearchString NVARCHAR(MAX),</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@SearchType VARCHAR(</code><code style="color: #009900;">4</code><code style="color: black;">),</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@Match BIT,</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@IgnoreCase BIT,</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@SearchSQLMetadata CHAR(</code><code style="color: #009900;">1</code><code style="color: black;">),</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@SchemaName NVARCHAR(</code><code style="color: #009900;">50</code><code style="color: black;">),</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@ObjectlisttoSearch NVARCHAR(MAX),</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@SearchCollate NVARCHAR(</code><code style="color: #009900;">500</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">/*************************************************************************</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">=================</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">INPUT PARAMETERS:</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">=================</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@SearchString - String to be searched</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@SearchType - ES - Exact Search using R</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #008200;">PS - Pattern Search using R</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #008200;">MPS - Multi Pattern Search - OR condition using R</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #008200;">NTLS - Normal T-SQL Like Search</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@Match - 0 = LIKE Search, 1 = NOT LIKE Search</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@IgnoreCase - 1 = case insensitive search, 0 = Case sensitive search (If @IgnoreCase IS NULL then default : case insensitive search)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@SearchSQLMetadata - Search sql definitions for presence of input string. 1 = Search, 0 = Don't Search</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@SchemaName - List of objects to be searched that fall under schema (Multiple schema's can be passed, separated by Comma)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@ObjectlisttoSearch - List of objects to be searched (Multiple table's can be passed, separated by Comma)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">--IF BOTH @ObjectlisttoSearch & @SchemaName ARE BLANK THEN ENTIRE DATABASE IS SEARCHED INCLUDING SQL DEFINITIONS</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">@SearchCollate - For @SearchType = NTLS if @IgnoreCase = 0. To search based on particular collation, default - COLLATE Latin1_General_CS_AS</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">*****************************************************************************/</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">AS</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">BEGIN</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SET NOCOUNT ON;</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF @SearchType IN (</code><code style="color: blue;">'ES'</code><code style="color: black;">,</code><code style="color: blue;">'PS'</code><code style="color: black;">,</code><code style="color: blue;">'MPS'</code><code style="color: black;">,</code><code style="color: blue;">'NTLS'</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">BEGIN</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">DECLARE @ExecutedBy NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">) = CURRENT_USER</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">DECLARE @Serv NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">) = CONCAT(CHAR(</code><code style="color: #009900;">39</code><code style="color: black;">),CHAR(</code><code style="color: #009900;">39</code><code style="color: black;">),@@SERVERNAME,CHAR(</code><code style="color: #009900;">39</code><code style="color: black;">),CHAR(</code><code style="color: #009900;">39</code><code style="color: black;">))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF ISNULL(@SchemaName,</code><code style="color: blue;">''</code><code style="color: black;">) <> </code><code style="color: blue;">''</code> <code style="color: black;">OR ISNULL(@ObjectlisttoSearch,</code><code style="color: blue;">''</code><code style="color: black;">) <> </code><code style="color: blue;">''</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">BEGIN</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">/**** List of table columns to be searched ****/</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">DECLARE @TableColList TABLE (Cols NVARCHAR(MAX),colname NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),Tbl NVARCHAR(</code><code style="color: #009900;">128</code><code style="color: black;">),TblCol NVARCHAR(</code><code style="color: #009900;">100</code><code style="color: black;">),ColType NVARCHAR(</code><code style="color: #009900;">150</code><code style="color: black;">))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">INSERT @TableColList</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">CASE WHEN TY.name IN (</code><code style="color: blue;">'date'</code><code style="color: black;">,</code><code style="color: blue;">'datetime2'</code><code style="color: black;">,</code><code style="color: blue;">'datetimeoffset'</code><code style="color: black;">,</code><code style="color: blue;">'time'</code><code style="color: black;">,</code><code style="color: blue;">'timestamp'</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">THEN CONCAT(</code><code style="color: blue;">'TRY_CONVERT('</code><code style="color: black;">,</code><code style="color: blue;">'VARCHAR(MAX),'</code><code style="color: black;">,C.name,</code><code style="color: blue;">') AS '</code><code style="color: black;">,QUOTENAME(C.NAME))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ELSE C.name END Columns -- To cover poor data type conversions when passed to R dataframe</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,C.name</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code style="color: black;">,T.name) TableName</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code style="color: black;">,T.name,</code><code style="color: blue;">'.'</code><code style="color: black;">,C.name) TblCol</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,TY.name</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">FROM Sys.tables T</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">JOIN sys.columns C</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">ON T.object_id = C.object_id</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">JOIN sys.types TY</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">ON C.<code>[</code>user_type_id] = TY.<code>[</code>user_type_id]</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">-- Ignore the datatypes that are not required</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">WHERE TY.name NOT IN (</code><code style="color: blue;">'geography'</code><code style="color: black;">,</code><code style="color: blue;">'varbinary'</code><code style="color: black;">,</code><code style="color: blue;">'binary'</code><code style="color: black;">,</code><code style="color: blue;">'text'</code><code style="color: black;">, </code><code style="color: blue;">'ntext'</code><code style="color: black;">, </code><code style="color: blue;">'image'</code><code style="color: black;">, </code><code style="color: blue;">'hierarchyid'</code><code style="color: black;">, </code><code style="color: blue;">'xml'</code><code style="color: black;">, </code><code style="color: blue;">'sql_variant'</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">AND (Schema_name(T.schema_id) IN (SELECT value FROM STRING_SPLIT(@SchemaName, </code><code style="color: blue;">','</code><code style="color: black;">))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">OR CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code style="color: black;">,T.name) IN (SELECT value FROM STRING_SPLIT(@ObjectlisttoSearch, </code><code style="color: blue;">','</code><code style="color: black;">)))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">END ELSE</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">BEGIN</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">INSERT @TableColList</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">CASE WHEN TY.name IN (</code><code style="color: blue;">'date'</code><code style="color: black;">,</code><code style="color: blue;">'datetime2'</code><code style="color: black;">,</code><code style="color: blue;">'datetimeoffset'</code><code style="color: black;">,</code><code style="color: blue;">'time'</code><code style="color: black;">,</code><code style="color: blue;">'timestamp'</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">THEN CONCAT(</code><code style="color: blue;">'TRY_CONVERT('</code><code style="color: black;">,</code><code style="color: blue;">'VARCHAR(MAX),'</code><code style="color: black;">,C.name,</code><code style="color: blue;">') AS '</code><code style="color: black;">,QUOTENAME(C.NAME))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ELSE C.name END Columns -- To cover poor data type conversions when passed to R dataframe</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,C.name</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code style="color: black;">,T.name) TableName</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,CONCAT(SCHEMA_NAME(T.SCHEMA_ID),</code><code style="color: blue;">'.'</code><code style="color: black;">,T.name,</code><code style="color: blue;">'.'</code><code style="color: black;">,C.name) TblCol</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,TY.name</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">FROM Sys.tables T</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">JOIN sys.columns C</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">ON T.object_id = C.object_id</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">JOIN sys.types TY</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">ON C.<code>[</code>user_type_id] = TY.<code>[</code>user_type_id]</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">-- Ignore the datatypes that are not required</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">WHERE TY.name NOT IN (</code><code style="color: blue;">'geography'</code><code style="color: black;">,</code><code style="color: blue;">'varbinary'</code><code style="color: black;">,</code><code style="color: blue;">'binary'</code><code style="color: black;">,</code><code style="color: blue;">'text'</code><code style="color: black;">, </code><code style="color: blue;">'ntext'</code><code style="color: black;">, </code><code style="color: blue;">'image'</code><code style="color: black;">, </code><code style="color: blue;">'hierarchyid'</code><code style="color: black;">, </code><code style="color: blue;">'xml'</code><code style="color: black;">, </code><code style="color: blue;">'sql_variant'</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">DROP TABLE IF EXISTS #ExportTablesList</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">CREATE TABLE #ExportTablesList (Rn BIGINT IDENTITY(</code><code style="color: #009900;">1</code><code style="color: black;">,</code><code style="color: #009900;">1</code><code style="color: black;">),cols NVARCHAR(</code><code style="color: #009900;">500</code><code style="color: black;">),colname NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),tbl NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),ColType NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF @SearchSQLMetadata = </code><code style="color: #009900;">1</code> <code style="color: black;">OR (@SearchSQLMetadata <> </code><code style="color: #009900;">0</code> <code style="color: black;">AND (ISNULL(@SchemaName,</code><code style="color: blue;">''</code><code style="color: black;">) = </code><code style="color: blue;">''</code> <code style="color: black;">AND ISNULL(@ObjectlisttoSearch,</code><code style="color: blue;">''</code><code style="color: black;">) = </code><code style="color: blue;">''</code><code style="color: black;">))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">BEGIN</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">INSERT #ExportTablesList (cols,tbl,ColType) SELECT</code><code style="color: blue;">'CONCAT('</code><code style="color: blue;">'<'</code><code style="color: blue;">',object_schema_name(sm.object_id),'</code><code style="color: blue;">'.'</code><code style="color: blue;">',object_name(sm.object_id),'</code><code style="color: blue;">'|'</code><code style="color: blue;">',o.type_desc COLLATE Latin1_General_100_CI_AS,'</code><code style="color: blue;">'>'</code><code style="color: blue;">',sm.definition) AS definition'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,</code><code style="color: blue;">'sys.sql_modules AS sm JOIN sys.objects AS o ON sm.object_id = o.object_id'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,</code><code style="color: blue;">'sql_modules'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">--Deduplication of object list</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">;WITH dedup</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">AS</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">(</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT *,ROW_NUMBER()OVER(PARTITION BY Tbl,Cols ORDER BY Cols) Rn FROM @TableColList</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">INSERT INTO #ExportTablesList</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT cols,colname,tbl,ColType FROM dedup</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">WHERE Rn = </code><code style="color: #009900;">1</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">AND tbl <> </code><code style="color: blue;">'dbo.Tbl_SearchString'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #008200;">/**** List of table columns to be searched ****/</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">IF (SELECT COUNT(</code><code style="color: #009900;">1</code><code style="color: black;">) FROM #ExportTablesList) <> </code><code style="color: #009900;">0</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">BEGIN</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">--Table to hold search output</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF NOT EXISTS (SELECT </code><code style="color: #009900;">1</code> <code style="color: black;">FROM sys.tables WHERE name = </code><code style="color: blue;">'Tbl_SearchString'</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">BEGIN</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">CREATE TABLE <code>[</code>dbo].<code>[</code>Tbl_SearchString] (</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>RunId] FLOAT,</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>SearchIndex] BIGINT,</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>SearchValue] NVARCHAR(MAX),</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>NoOfOccurance] FLOAT,</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>ObjectName] NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>ColumnNameORDefinition] NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>SqlDatatype] NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>InputParameter] NVARCHAR(</code><code style="color: #009900;">800</code><code style="color: black;">),</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>ExecutedBy] NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;"><code>[</code>ExecutedAt] DATETIME</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">DECLARE @RunId FLOAT</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT @RunId = COALESCE(MAX(<code>[</code>RunId]),</code><code style="color: #009900;">0</code><code style="color: black;">)+</code><code style="color: #009900;">1</code> <code style="color: black;">FROM <code>[</code>dbo].<code>[</code>Tbl_SearchString]</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">--Processing to store input parameters</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">DECLARE @Input NVARCHAR(MAX) = CONCAT(</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: blue;">'@SearchString > '</code> <code style="color: black;">,CASE WHEN @SearchString = </code><code style="color: blue;">''</code> <code style="color: black;">OR @SearchString IS NULL THEN </code><code style="color: blue;">'NULL'</code> <code style="color: black;">ELSE @SearchString END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">,</code><code style="color: blue;">',@SearchType > '</code> <code style="color: black;">,CASE WHEN @SearchType = </code><code style="color: blue;">''</code> <code style="color: black;">OR @SearchType IS NULL THEN </code><code style="color: blue;">'NULL'</code> <code style="color: black;">ELSE @SearchType END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">,</code><code style="color: blue;">',@Match > '</code> <code style="color: black;">,COALESCE(@Match,</code><code style="color: #009900;">0</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">,</code><code style="color: blue;">',@IgnoreCase > '</code> <code style="color: black;">,COALESCE(@IgnoreCase,</code><code style="color: #009900;">1</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">,</code><code style="color: blue;">',@SearchSQLMetadata > '</code> <code style="color: black;">,CASE WHEN @SearchSQLMetadata = </code><code style="color: blue;">''</code> <code style="color: black;">OR @SearchSQLMetadata IS NULL THEN </code><code style="color: blue;">'NULL'</code> <code style="color: black;">ELSE @SearchSQLMetadata END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">,</code><code style="color: blue;">',@SchemaName > '</code> <code style="color: black;">,CASE WHEN @SchemaName = </code><code style="color: blue;">''</code> <code style="color: black;">OR @SchemaName IS NULL THEN </code><code style="color: blue;">'NULL'</code> <code style="color: black;">ELSE @SchemaName END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">,</code><code style="color: blue;">',@ObjectlisttoSearch > '</code> <code style="color: black;">,CASE WHEN @ObjectlisttoSearch = </code><code style="color: blue;">''</code> <code style="color: black;">OR @ObjectlisttoSearch IS NULL THEN </code><code style="color: blue;">'NULL'</code> <code style="color: black;">ELSE @ObjectlisttoSearch END)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">--By </code><code style="color: #009900;">default</code> <code style="color: black;">case insensitive search</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT @IgnoreCase = COALESCE(@IgnoreCase,</code><code style="color: #009900;">1</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">--By </code><code style="color: #009900;">default</code> <code style="color: black;">LIKE search</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT @Match = COALESCE(@Match,</code><code style="color: #009900;">0</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF @SearchType = </code><code style="color: blue;">'NTLS'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">BEGIN</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">DECLARE @SearchStrings TABLE (Id INT IDENTITY(</code><code style="color: #009900;">1</code><code style="color: black;">,</code><code style="color: #009900;">1</code><code style="color: black;">),String NVARCHAR(MAX))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">INSERT @SearchStrings</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT value FROM STRING_SPLIT(@SearchString, </code><code style="color: blue;">'|'</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">UPDATE #ExportTablesList SET Tbl = </code><code style="color: blue;">'sys.sql_modules'</code><code style="color: black;">, colname = </code><code style="color: blue;">'definition'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">WHERE ColType = </code><code style="color: blue;">'sql_modules'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SET @SearchCollate = CASE WHEN @SearchCollate = </code><code style="color: blue;">''</code> <code style="color: black;">THEN NULL ELSE @SearchCollate END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">DECLARE @COLLATE NVARCHAR(</code><code style="color: #009900;">100</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SET @COLLATE = CASE WHEN @IgnoreCase = </code><code style="color: #009900;">0</code> <code style="color: black;">THEN CASE WHEN @SearchCollate = </code><code style="color: blue;">''</code> <code style="color: black;">OR @SearchCollate IS NULL THEN </code><code style="color: blue;">' COLLATE Latin1_General_CS_AS '</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ELSE CONCAT(</code><code style="color: blue;">' COLLATE '</code><code style="color: black;">,@SearchCollate,</code><code style="color: blue;">' '</code><code style="color: black;">) END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ELSE CHAR(</code><code style="color: #009900;">32</code><code style="color: black;">) END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">DECLARE @SearchOperator NVARCHAR(</code><code style="color: #009900;">100</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SET @SearchOperator = CASE WHEN @Match = </code><code style="color: #009900;">1</code> <code style="color: black;">THEN </code><code style="color: blue;">' NOT LIKE '</code> <code style="color: black;">ELSE </code><code style="color: blue;">' LIKE '</code> <code style="color: black;">END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">DECLARE @WHEREClause NVARCHAR(MAX)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">;WITH CTE</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">AS</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">(</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT </code><code style="color: blue;">'SearchValue '</code><code style="color: black;">+ @SearchOperator +</code><code style="color: blue;">''</code><code style="color: blue;">''</code><code style="color: black;">+String+</code><code style="color: blue;">''</code><code style="color: blue;">''</code><code style="color: black;">+@COLLATE WhereClause FROM @SearchStrings</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SELECT @WHEREClause = STUFF(</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">(SELECT </code><code style="color: blue;">' OR '</code> <code style="color: black;">+ WhereClause FROM</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">(SELECT WhereClause FROM CTE ) AS T FOR XML PATH(</code><code style="color: blue;">''</code><code style="color: black;">)),</code><code style="color: #009900;">2</code><code style="color: black;">,</code><code style="color: #009900;">2</code><code style="color: black;">,</code><code style="color: blue;">''</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SET @SearchString = CASE WHEN @SearchType = </code><code style="color: blue;">'ES'</code> <code style="color: black;">THEN REPLACE(@SearchString,</code><code style="color: blue;">'"'</code><code style="color: black;">,</code><code style="color: blue;">''</code><code style="color: black;">) ELSE @SearchString END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #008200;">/**** Loop through above Objects list and execute R script ****/</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">DECLARE @I INT = </code><code style="color: #009900;">1</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@SQL NVARCHAR(MAX) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@RScript NVARCHAR(MAX) = N</code><code style="color: blue;">''</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@tblname NVARCHAR(</code><code style="color: #009900;">128</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Colname NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Sqltype NVARCHAR(</code><code style="color: #009900;">100</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">WHILE @I <= (SELECT MAX(Rn) FROM #ExportTablesList)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">BEGIN</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SELECT @SQL = CONCAT(</code><code style="color: blue;">'SELECT '</code><code style="color: black;">,Cols,</code><code style="color: blue;">' FROM '</code><code style="color: black;">,tbl)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@tblname = Tbl</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Colname = CASE WHEN @SearchType IN (</code><code style="color: blue;">'ES'</code><code style="color: black;">,</code><code style="color: blue;">'PS'</code><code style="color: black;">) THEN cols ELSE colname END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Sqltype = ColType</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">FROM #ExportTablesList WHERE Rn = @I</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF @SearchType IN (</code><code style="color: blue;">'ES'</code><code style="color: black;">,</code><code style="color: blue;">'PS'</code><code style="color: black;">,</code><code style="color: blue;">'MPS'</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">BEGIN</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SET @RScript = '</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">#Provide DB credential detail for storing output in a table</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">sqlConnString <- </code><code style="color: blue;">"Driver=SQL Server;Server=serv; Database=WideWorldImporters;Uid=sa;Pwd=password"</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">#function to count no of occurences</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">countCharOccurrences <- function(char,string,Type) {</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">if (Type ==</code><code style="color: blue;">"ES"</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">{</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Boundchar <- paste</code><code style="color: #009900;">0</code><code style="color: black;">(</code><code style="color: blue;">"\\b"</code><code style="color: black;">,char,</code><code style="color: blue;">"\\b"</code><code style="color: black;">,sep =</code><code style="color: blue;">""</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">string</code><code style="color: #009900;">1</code> <code style="color: black;"><- gsub(Boundchar,</code><code style="color: blue;">""</code><code style="color: black;">,string,ignore.case=IgnoreCase)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">}</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">string</code><code style="color: #009900;">1</code> <code style="color: black;"><- gsub(char,</code><code style="color: blue;">""</code><code style="color: black;">,string,ignore.case=IgnoreCase)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">return ((nchar(string) - nchar(string</code><code style="color: #009900;">1</code><code style="color: black;">))/nchar(char))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">}</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">#getting input dataset column name into a variable </code><code style="color: blue;">"c"</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">c <- colnames(InputDataSet)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">if (SearchType == </code><code style="color: blue;">"ES"</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">{</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ExactString <- paste</code><code style="color: #009900;">0</code><code style="color: black;">(</code><code style="color: blue;">"\\b"</code><code style="color: black;">,SearchString,</code><code style="color: blue;">"\\b"</code><code style="color: black;">,sep =</code><code style="color: blue;">""</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output <- as.data.frame(grep(ExactString,InputDataSet<code>[</code><code>[</code>c]],ignore.case = IgnoreCase,</code><code style="color: #009900;">invert</code> <code style="color: black;">= Match))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">colnames(Output)<code>[</code></code><code style="color: #009900;">1</code><code style="color: black;">] <- </code><code style="color: blue;">"SearchIndex"</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$SearchValue <- grep(ExactString,InputDataSet<code>[</code><code>[</code>c]],ignore.case = IgnoreCase,value = TRUE,</code><code style="color: #009900;">invert</code> <code style="color: black;">= Match)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$NoOfOccurance <- countCharOccurrences(SearchString,Output$SearchValue,SearchType)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">}</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">if (SearchType == </code><code style="color: blue;">"PS"</code> <code style="color: black;">|| SearchType == </code><code style="color: blue;">"MPS"</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">{</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output <- as.data.frame(grep(SearchString,InputDataSet<code>[</code><code>[</code>c]],ignore.case = IgnoreCase,</code><code style="color: #009900;">invert</code> <code style="color: black;">= Match))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">colnames(Output)<code>[</code></code><code style="color: #009900;">1</code><code style="color: black;">] <- </code><code style="color: blue;">"SearchIndex"</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$SearchValue <- grep(SearchString,InputDataSet<code>[</code><code>[</code>c]],ignore.case = IgnoreCase,value = TRUE,</code><code style="color: #009900;">invert</code> <code style="color: black;">= Match)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">if (SearchType == </code><code style="color: blue;">"PS"</code><code style="color: black;">) {</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$NoOfOccurance <- countCharOccurrences(SearchString,Output$SearchValue,SearchType) }</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">}</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$ObjectName <- rep(tblname,nrow(Output))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$ColumnNameORDefinition <- rep(c,nrow(Output))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$SqlDatatype <- rep(Sqltype,nrow(Output))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$ObjectName<code>[</code>Output$SqlDatatype == </code><code style="color: blue;">"sql_modules"</code><code style="color: black;">] <- </code><code style="color: blue;">"sql_modules"</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$InputParameter <- rep(Input,nrow(Output))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$ExecutedBy <- rep(ExecutedBy,nrow(Output))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$ExecutedAt <- rep(</code><code style="color: #009900;">format</code><code style="color: black;">(Sys.time(),usetz = FALSE),nrow(Output))</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">Output$RunId <- rep(RunId,nrow(Output))</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">sqlDS <- RxSqlServerData(connectionString = sqlConnString,table = </code><code style="color: blue;">"Tbl_SearchString"</code><code style="color: black;">)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">rxDataStep(inData = Output, outFile = sqlDS,append = </code><code style="color: blue;">"rows"</code><code style="color: black;">)</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">EXEC sp_execute_external_script</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">@language = N</code><code style="color: blue;">'R'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@script = @RScript</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@input_data_</code><code style="color: #009900;">1</code> <code style="color: black;">= @SQL</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@params = N'@SearchString NVARCHAR(MAX),@SearchType VARCHAR(</code><code style="color: #009900;">4</code><code style="color: black;">),@Match BIT,@IgnoreCase BIT,@Input NVARCHAR(MAX)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@tblname NVARCHAR(</code><code style="color: #009900;">128</code><code style="color: black;">),@Sqltype NVARCHAR(</code><code style="color: #009900;">150</code><code style="color: black;">),@ExecutedBy NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">),@RunId FLOAT</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Serv NVARCHAR(</code><code style="color: #009900;">200</code><code style="color: black;">)'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@SearchString = @SearchString</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@SearchType = @SearchType</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Match = @Match</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@IgnoreCase = @IgnoreCase</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Input = @Input</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@tblname = @tblname</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Sqltype = @Sqltype</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@ExecutedBy = @ExecutedBy</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@RunId = @RunId</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Serv = @Serv</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">IF @SearchType = </code><code style="color: blue;">'NTLS'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">BEGIN</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">INSERT <code>[</code>dbo].<code>[</code>Tbl_SearchString](<code>[</code>RunId],<code>[</code>SearchIndex],<code>[</code>SearchValue],<code>[</code>ObjectName]</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,<code>[</code>ColumnNameORDefinition],<code>[</code>SqlDatatype],<code>[</code>InputParameter],<code>[</code>ExecutedBy],<code>[</code>ExecutedAt])</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">EXEC (</code><code style="color: blue;">'SELECT '</code><code style="color: black;">+@RunId+</code><code style="color: blue;">',SearchIndex,SearchValue,'</code><code style="color: blue;">''</code><code style="color: black;">+@tblname+</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code style="color: black;">+@Colname+</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code style="color: black;">+@Sqltype+</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code style="color: black;">+@Input+</code><code style="color: blue;">''</code><code style="color: blue;">','</code><code style="color: blue;">''</code><code style="color: black;">+@ExecutedBy+</code><code style="color: blue;">''</code><code style="color: black;">', GETDATE()</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">FROM (SELECT ROW_NUMBER()OVER(ORDER BY (SELECT </code><code style="color: #009900;">1</code><code style="color: black;">)) SearchIndex,</code><code style="color: blue;">'+@Colname+'</code> <code style="color: black;">AS SearchValue FROM '+@tblname+</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: blue;">' ) Tmp WHERE '</code><code style="color: black;">+@WHEREClause)</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">END</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">SET @I = @I + </code><code style="color: #009900;">1</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: black;">END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #008200;">/**** Loop through above table list and execute R script ****/</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">--Display final search result</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SELECT * FROM <code>[</code>dbo].<code>[</code>Tbl_SearchString] WHERE RunId = @RunId AND ExecutedBy = CURRENT_USER</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ELSE</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SELECT </code><code style="color: blue;">'No valid objects passed in the InputParameter to search the string'</code> <code style="color: black;">AS InvalidParameter</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">ELSE</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">SELECT 'SearchType parameter is mandatory ES - Exact Search, PS - Pattern Search,MPS - Multi Pattern Search - OR condition</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,NTLS - Normal T-SQL Like Search' AS InvalidParameter</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code> </div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">END</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<span style="font-size: 12.1104px;">Sample execution code block:</span></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<span style="font-size: 12.1104px;"><br /></span></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">EXEC</code> <code style="color: black;">usp_SearchString @SearchString = </code><code style="color: blue;">'VAT'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@SearchType = </code><code style="color: blue;">'ES'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@Match = 0 </code><code style="color: #008200;">-- 0 = LIKE, 1 = NOT LIKE</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@IgnoreCase = 1 </code><code style="color: #008200;">-- 1 = Case insensitive, 0 = Case Sensitive</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@SearchSQLMetadata= 0 </code><code style="color: #008200;">-- 1 = Search, 0 = Don't Search</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,@SchemaName = '</code><code style="color: blue;">'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: blue;">,@ObjectlisttoSearch = '</code><code style="color: black;">dbo.Tmp</code><code style="color: blue;">'</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: blue;">,@SearchCollate = '</code><code style="color: black;">'</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h1 style="background-color: white; clear: both; color: #707070; font-family: "Segoe UI Light", "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 36.3312px; font-weight: 100; line-height: 1.3;">
Sample script execution screenshots</h1>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
<a href="https://www.blogger.com/null" name="Search_string_in_table_from_DB" style="color: #00749e; cursor: pointer;"></a>1. Search string in table from DB </h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Example 1: If we want to search for a string "Ava" on Application.People table from WideWorldImporters database, we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRDUs719Jl8TKRkCywK18DOa3-ORJ6lihbc1eGNJ52ZTzCOQkYZdNPjJ_tuIwSgW7_WHXl-niURx7yG6vkfLNIodESLOccYOoXdIa3DL8yWPcTVbbG6TuJMSxMrefqkm3SF8QZvI9Utj3I/s1600/SS1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="597" data-original-width="1358" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRDUs719Jl8TKRkCywK18DOa3-ORJ6lihbc1eGNJ52ZTzCOQkYZdNPjJ_tuIwSgW7_WHXl-niURx7yG6vkfLNIodESLOccYOoXdIa3DL8yWPcTVbbG6TuJMSxMrefqkm3SF8QZvI9Utj3I/s640/SS1.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
2. Exact search in table from DB</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
In example 1, we did a pattern search. If we want to do a exact search for a string "Ava" on Application.People table from WideWorldImporters database, we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaI6sIlaN3zLuFauY9YJRTt3w1FivDyhTdx99GdS9xf9cnvtY5tf0qvFcPfODhFkwzdNOZcxD69q-2El2BFmiD65nn8JAyl62qFc5lWF2M-KA8R46W_3KQnl2VYCujuNxC1AWQNsPQNYPx/s1600/SS2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="598" data-original-width="1361" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaI6sIlaN3zLuFauY9YJRTt3w1FivDyhTdx99GdS9xf9cnvtY5tf0qvFcPfODhFkwzdNOZcxD69q-2El2BFmiD65nn8JAyl62qFc5lWF2M-KA8R46W_3KQnl2VYCujuNxC1AWQNsPQNYPx/s640/SS2.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
3. Exact case sensitive search</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Example 3: In example 2, we did a exact search. If we want to do a exact case sensitive search for a string "Ava" on Application.People table from WideWorldImporters database, we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9i5aNJB_ufI4TLGChF_8gsfNJozIDD-5cCG95IpGYzWgNX5KY3uh1XIuiG1jwBfHTG2MSdVeYraGia0hBg31syZ8FVXY7lf-Frk3EL3H7KGbece1YxJHq0eS8OpsJBjwVuB1Wzsfd4cTX/s1600/SS3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="597" data-original-width="1361" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9i5aNJB_ufI4TLGChF_8gsfNJozIDD-5cCG95IpGYzWgNX5KY3uh1XIuiG1jwBfHTG2MSdVeYraGia0hBg31syZ8FVXY7lf-Frk3EL3H7KGbece1YxJHq0eS8OpsJBjwVuB1Wzsfd4cTX/s640/SS3.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
4. Exact case sensitive search (NTLS)</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
If we want to do a exact case sensitive search for a string "male" on Purchasing.PurchaseOrderLines table from WideWorldImporters database, we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhY3wjh2BA5VZuXEm-zv3uv2CmxoggEoepIbIEY9VnDQfQd5DVTRROvsbKKX5AM0jHoiwMizydAQx3mvOKL7xX50LtAkmiicJUcv3IYp0d37ap1_gm1AnNQbQfLqOOPWAS1XjjMyOPwb7my/s1600/SS4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="592" data-original-width="1361" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhY3wjh2BA5VZuXEm-zv3uv2CmxoggEoepIbIEY9VnDQfQd5DVTRROvsbKKX5AM0jHoiwMizydAQx3mvOKL7xX50LtAkmiicJUcv3IYp0d37ap1_gm1AnNQbQfLqOOPWAS1XjjMyOPwb7my/s640/SS4.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
5. Exact case sensitive search (Using R)</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
In example 4, it returned two rows (records that contains both "male" and "female") as the @SearchType was set NTLS (Normal T-SQL Like search). But we actually expect the result to be one row with record that contain only "male". If we again do a exact case sensitive search for a string "male" on Purchasing.PurchaseOrderLines table from WideWorldImporters database, by setting @SearchType = 'ES' shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglzp8PHSXIP9l9ZSgjD6k-2HVt7Kd24rths8g2hUOcPWPWnUbDvjdAEsHGshTh3JA6FgBhQohwnmkovdIuUmDZF5DNxLr_Z8MG5mm7rR57JaKEkXDpSlkMyFH7y_Wr1345WXt5zZpeR8WB/s1600/SS5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="595" data-original-width="1359" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglzp8PHSXIP9l9ZSgjD6k-2HVt7Kd24rths8g2hUOcPWPWnUbDvjdAEsHGshTh3JA6FgBhQohwnmkovdIuUmDZF5DNxLr_Z8MG5mm7rR57JaKEkXDpSlkMyFH7y_Wr1345WXt5zZpeR8WB/s640/SS5.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
6. Multi string search delimited by pipe</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
If we want to do a multi string search delimited by pipe (search string "Ava","Amy") on Application.People table from WideWorldImporters database, we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSqKEf9VGzCBN3v-Twl_rSGsDKd7pTnhaLMCD6wdTSTvHj_nePAzspVa72ivbPN_MKJ4oV4ATEVAAa4HrSYQ0CR6erojSZlyvQFgzWXwFSwzIGInf-9AIrwdHN1_Kqp0bzTJm-LZVh-LQI/s1600/SS6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="599" data-original-width="1362" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSqKEf9VGzCBN3v-Twl_rSGsDKd7pTnhaLMCD6wdTSTvHj_nePAzspVa72ivbPN_MKJ4oV4ATEVAAa4HrSYQ0CR6erojSZlyvQFgzWXwFSwzIGInf-9AIrwdHN1_Kqp0bzTJm-LZVh-LQI/s640/SS6.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Please note NoOfOccurance field will be populated only for @SearchType = "ES" & "PS" (for single string search without wildcard).</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
<a href="https://www.blogger.com/null" name="multi_string_search_using_normal_T-SQL_LIKE_search" style="color: #00749e; cursor: pointer;"></a>7. Multi string search using normal T-SQL LIKE search</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
In example 6, multi string search was done using R script. If we want to do the same multi string search using normal T-SQL LIKE search, we can try by setting parameters values as shown below :</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Also in this example, let us see how we can search string with specific collation setting.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjg5Zc6lK_4UOR5_DnxVTg8yO9DzFSjMRd_Yqmu0IdEhYneHq_NQ6OLIT90st97SlmPUUZeNGFpWgEhmoWuJijzk9VREL1kkCR9KO5Ci1cTwmDS7t3y4g3RBtf0ffo4gn73GWeUThtcoC9G/s1600/SS7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="598" data-original-width="1358" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjg5Zc6lK_4UOR5_DnxVTg8yO9DzFSjMRd_Yqmu0IdEhYneHq_NQ6OLIT90st97SlmPUUZeNGFpWgEhmoWuJijzk9VREL1kkCR9KO5Ci1cTwmDS7t3y4g3RBtf0ffo4gn73GWeUThtcoC9G/s640/SS7.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
8. Fixed pattern search</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
If we want to do fixed pattern search say if we know the string to be searched is a two letter alphabet then we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMuZ0AvbovnCUiqlC9Hj8Bpj59fnfl0j7XVUFnJTw7V1RPhyphenhyphen3z3-kiOHhvp0KPAtH6gKNuXrOcRuOgsEY3DI0JgksAMDItKcKQTNAY8ef9XNnaKPfR0ncTxMrNceP55QlWEbAVAWhZPE5D/s1600/SS8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="599" data-original-width="1365" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMuZ0AvbovnCUiqlC9Hj8Bpj59fnfl0j7XVUFnJTw7V1RPhyphenhyphen3z3-kiOHhvp0KPAtH6gKNuXrOcRuOgsEY3DI0JgksAMDItKcKQTNAY8ef9XNnaKPfR0ncTxMrNceP55QlWEbAVAWhZPE5D/s640/SS8.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
9. Search phrase from multiple tables delimited by comma</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Example 9: If we want to search for a phrase from multiple tables delimited by comma then we can try by setting parameters values as shown below :<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8PCCFgAXxWT1Kj9UUodpSkugNpkcxwTZQhHZE32b5AMC2r66tnCRri036Rn4p30K1HHHw5DCTE1Kg6n7Vho6_yPwd8HtdcBEmAD5pXhi0lRMLPVRnjAYFjf9hCVNHXrugHD26er4kJbQM/s1600/SS9.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="603" data-original-width="1361" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8PCCFgAXxWT1Kj9UUodpSkugNpkcxwTZQhHZE32b5AMC2r66tnCRri036Rn4p30K1HHHw5DCTE1Kg6n7Vho6_yPwd8HtdcBEmAD5pXhi0lRMLPVRnjAYFjf9hCVNHXrugHD26er4kJbQM/s640/SS9.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
10. Search for a date from multiple schema's delimited by comma</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Example 10: If we want to search for a date from multiple schema's delimited by comma then we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXdPVsuvDYkNro2nh0rdv3pJABgqQt0xvaAigHl2NvsaBdlFw2g3Dg4iBBCHWGuT2Lq-ZPkw8y7V8eNyuHKYnF38uqv5uxJbz3rIcztlHNH6W8wQZyK3BtcZqFoHMqmbwFUpMAG2nXl2mV/s1600/SS10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="595" data-original-width="1361" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXdPVsuvDYkNro2nh0rdv3pJABgqQt0xvaAigHl2NvsaBdlFw2g3Dg4iBBCHWGuT2Lq-ZPkw8y7V8eNyuHKYnF38uqv5uxJbz3rIcztlHNH6W8wQZyK3BtcZqFoHMqmbwFUpMAG2nXl2mV/s640/SS10.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
11. Search string called "Password" in entire database</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Example 11: If we want to search for a string called "Password" in entire database including SQL object definitions then we can try by setting parameters values as shown below :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimSSk9JU0SLrF5T9_snTuzo6ozVmznzGBtsSjW9f3XZOy4FGXvmPOMZScA8n2x3eyuAUui8WKtS0tyYAGspDn0pM8Ztg_W0l5ow6KMUIrEyuqpIFU2H-Qs8O3E6q5ik1agPdquZFukZVD9/s1600/SS11.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="597" data-original-width="1360" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimSSk9JU0SLrF5T9_snTuzo6ozVmznzGBtsSjW9f3XZOy4FGXvmPOMZScA8n2x3eyuAUui8WKtS0tyYAGspDn0pM8Ztg_W0l5ow6KMUIrEyuqpIFU2H-Qs8O3E6q5ik1agPdquZFukZVD9/s640/SS11.JPG" width="640" /></a></div>
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<h2 style="background-color: white; border-bottom: 0px; clear: both; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 19.9822px; font-weight: normal; line-height: 1.25; margin: 0px;">
12. Wild card search</h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
Example 12: Below example shows how we can do wildcard search when the search is done using R script, refer parameters values as shown below :</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
To know more about R wild card search using "?","*","^","$", please see the link provided in reference section.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvcNsVcHU0xMrsvY0yOGaQUijZ-eY-2gYJfu3phtrQNhpcHZ-hWP7L_9KJgNSMmmHnBlJaLL964ASi2hp64NbdfooXRRNqoeVlePojhwTwAOU0M8sn62PBnGydPfHUBx7kw0fdzqGuC9k3/s1600/SS12.JPG" imageanchor="1" style="font-size: 12.1104px; margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="596" data-original-width="1361" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvcNsVcHU0xMrsvY0yOGaQUijZ-eY-2gYJfu3phtrQNhpcHZ-hWP7L_9KJgNSMmmHnBlJaLL964ASi2hp64NbdfooXRRNqoeVlePojhwTwAOU0M8sn62PBnGydPfHUBx7kw0fdzqGuC9k3/s640/SS12.JPG" width="640" /></a></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<span style="font-size: 12.1104px;">Tbl_SearchString stores the details about search made on a string. If we want to see the entire details (all the other fields from a table) of the record that matches the searched string then we can try like shown below:</span><br />
<span style="font-size: 12.1104px;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoWbgZJpL2Z80ynlRprEtmEBcTEZkxd0S2CDcFS9XFFRv_929q7Z_0YldA8ciwzxHjuIOyCNgukXAU9NIyIlJk5mcvUfzLsblTpQ-sweDgi4UbRTJqj_p8j4cGPTZAxrNML60Sv30bCTXu/s1600/SS13.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="595" data-original-width="1353" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoWbgZJpL2Z80ynlRprEtmEBcTEZkxd0S2CDcFS9XFFRv_929q7Z_0YldA8ciwzxHjuIOyCNgukXAU9NIyIlJk5mcvUfzLsblTpQ-sweDgi4UbRTJqj_p8j4cGPTZAxrNML60Sv30bCTXu/s640/SS13.JPG" width="640" /></a></div>
<span style="font-size: 12.1104px;"><br /></span></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<span style="font-size: 12.1104px;"><br /></span></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;">--To get deails of particular RunId</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: #006699; font-weight: bold;">DISTINCT</code> <code style="color: black;">RunId</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,<code>[</code>ObjectName]</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;"><code>[</code>WideWorldImporters].<code>[</code>dbo].<code>[</code>Tbl_SearchString]</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">WHERE</code> <code style="color: black;">RunId = 12</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
</div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">A.*</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: black;">,B.*</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;"><code>[</code>WideWorldImporters].<code>[</code>dbo].<code>[</code>Tbl_SearchString] A</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: grey;">JOIN</code> <code style="color: black;">(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: black;">row_number()over(</code><code style="color: #006699; font-weight: bold;">order</code> <code style="color: #006699; font-weight: bold;">by</code> <code style="color: black;">(</code><code style="color: #006699; font-weight: bold;">SELECT</code> <code style="color: blue;">'A'</code><code style="color: black;">)) Rn,* </code><code style="color: #006699; font-weight: bold;">FROM</code> <code style="color: black;">Warehouse.StockItemHoldings ) B </code><code style="color: #008200;">--Change table name</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: #006699; font-weight: bold;">ON</code> <code style="color: black;">A.SearchIndex = B.Rn</code></div>
<div style="background-color: #f8f8f8; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: grey;">AND</code> <code style="color: black;">A.ObjectName = </code><code style="color: blue;">'Warehouse.StockItemHoldings'</code> <code style="color: #008200;">--Change table name</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code> </code><code style="color: grey;">AND</code> <code style="color: black;">RunId = 12 </code><code style="color: #008200;">--provide run id</code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<code style="color: #008200;"><br /></code></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br /></div>
<h2 style="background-color: white; color: #2a2a2a; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>See Also:</i></span></h2>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
<br /></div>
<div style="background-color: white; color: #2a2a2a; font-family: "Segoe UI", "Lucida Grande", Verdana, Arial, Helvetica, sans-serif; font-size: 12.1104px;">
</div>
<ul style="text-align: left;">
<li><i>My post on Technet wiki - <a href="https://social.technet.microsoft.com/wiki/contents/articles/52793.t-sql-search-for-string-or-phrase-in-sql-server-database.aspx" target="_blank">https://social.technet.microsoft.com/wiki/contents/articles/52793.t-sql-search-for-string-or-phrase-in-sql-server-database.aspx</a></i></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-46571404976951204132019-01-06T21:19:00.001+05:302019-07-06T01:07:27.104+05:30Import csv or text file into SQL Server table<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how to import csv or text file into SQL Server table. If you want to import some random files on adhoc basis then we can make use of<a href="https://docs.microsoft.com/en-us/sql/relational-databases/import-export/import-flat-file-wizard?view=sql-server-2017" target="_blank"> <i>Import Flat File Wizard from SQL Server Management Studio</i></a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If the requirement is to do on regular basis or import multiple files simultaneously then we can do this using t-sql script that executes R script to read an csv or text file & load into SQL Server table.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">As a Pre-requisites, </span><a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/install/sql-r-services-windows-install?view=sql-server-2017" style="background-color: white; color: #00749e; cursor: pointer; outline: none; text-decoration-line: none;" target="_blank">Install R services <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: auto !important; max-width: 100%; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a><span style="background-color: white;">and then from SSMS enable the external scripting feature. Restart the database engine and then verify the installation as mentioned in MSDN.</span></span><br />
<div style="background-color: white; color: #2a2a2a;">
<br /></div>
<div style="background-color: white; color: #2a2a2a;">
<span style="color: black;"><span style="font-family: "arial" , "helvetica" , sans-serif;">You might face “Access is denied” error while executing below scripts, to overcome that right-click on the folder where CSV files will be imported / exported and provide folder access (read, write) to "ALL_APPLICATION_PACKAGES".</span></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In my previous post on <a href="http://www.allaboutmssql.com/2019/01/sql-server-export-adhoc-sql-or-table-to.html" target="_blank"><i>SQL Server export adhoc SQL or table to CSV file</i></a>, I have exported few csv files. I am going to make use of those three csv files for this example.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivnVKLIf87ztzx-PvYAbdTa2Q2Ov5Orv6G_aWNrZEq0oxz2CwfBbG72Csq5wA-avej1Dvp3aWUpIY5boVFAFXMbVL5neothRfXpdwAj2q54NLqSd_gtW5mES4aFo3DfJpIGGpCqc29QjxK/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="191" data-original-width="658" height="184" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivnVKLIf87ztzx-PvYAbdTa2Q2Ov5Orv6G_aWNrZEq0oxz2CwfBbG72Csq5wA-avej1Dvp3aWUpIY5boVFAFXMbVL5neothRfXpdwAj2q54NLqSd_gtW5mES4aFo3DfJpIGGpCqc29QjxK/s640/1.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;">I have also created a database called "Staging" to load the csv files into different SQL tables created dynamically with csv filename as table name.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below screenshot shows executing the below script imports the csv files as SQL Server tables under staging database.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3rZNRCyntX9mqkR1mbYVVUgEud0bwIgbhFlI-NpNuZrLi6s72UIpVnxbuLuGA1qbqZdyU1V0_SLYDGM246mpwx7k9poOvOSDjr6M7_AwvS9LOe8Iuc-8a9CpgY5ExF0lDpuyQlXsSR7ll/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="627" data-original-width="1243" height="322" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3rZNRCyntX9mqkR1mbYVVUgEud0bwIgbhFlI-NpNuZrLi6s72UIpVnxbuLuGA1qbqZdyU1V0_SLYDGM246mpwx7k9poOvOSDjr6M7_AwvS9LOe8Iuc-8a9CpgY5ExF0lDpuyQlXsSR7ll/s640/2.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Please note: In the below script, we need to configure the values for </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">sqlconnString variable with database connection details</span><br />
<span style="font-family: arial, helvetica, sans-serif;">list_filespath & </span><span style="font-family: arial, helvetica, sans-serif;">list_filenames </span><span style="font-family: arial, helvetica, sans-serif;">variable with folder path details</span><br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;">DECLARE @RScript NVARCHAR(MAX) =N' </span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> library(foreach);</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlConnString <- "Driver=SQL Server;Server=''LAPTOP-LCUB6HBB''; Database=Staging;Uid=sa;Pwd=***"</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> list_filespath <- list.files(path = "H:/Temp/Blog/ImportExportExcel", pattern = "*.csv", all.files = FALSE,</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> full.names = TRUE, recursive = TRUE,</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> ignore.case = FALSE, include.dirs = FALSE, no.. = TRUE)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> list_filenames <- list.files(path = "H:/Temp/Blog/ImportExportExcel", pattern = "*.csv", all.files = FALSE,</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> full.names = FALSE, recursive = TRUE,</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> ignore.case = FALSE, include.dirs = FALSE, no.. = TRUE)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> #Check file exists in path</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> if (length(list_filespath) != 0) {</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> foreach(m = 1:length(list_filespath)) %do%</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> {</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlTable <- strsplit(list_filenames[m],split = ".csv") </span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlTable <- gsub("\\s", "", sqlTable)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlTable <- gsub("[^[:alnum:]]", "", sqlTable)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> Output <- read.csv(file=list_filespath[m], header=TRUE, sep=",")</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlDS <- RxSqlServerData(connectionString = sqlConnString,table = sqlTable)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> rxDataStep(inData = Output, outFile = sqlDS,overwrite = TRUE)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> }</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> }'</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;">EXEC sp_execute_external_script</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> @language = N'R'</span><br />
<br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> ,@script = @RScript</span><br />
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>See Also:</i></span></h3>
<div style="text-align: left;">
<br />
<ul style="text-align: left;">
<li><i style="font-family: arial, helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_59.html" target="_blank">SQL Server Export/Import excel using R script</a></i></li>
<li><a href="https://social.technet.microsoft.com/wiki/contents/articles/52484.sql-server-importexport-to-csv-using-r-script.aspx" style="font-family: arial, helvetica, sans-serif;" target="_blank"><i>My post on Technet wiki - SQL Server Import/Export to CSV using R script</i></a></li>
</ul>
</div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-33827432154236409792019-01-01T03:11:00.005+05:302019-07-06T00:59:02.195+05:30SQL Server export adhoc SQL or table to CSV file<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how to export adhoc sql query output or list of tables, views data into csv file. I am going to make use of R script to do this.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">I have tried this example with SQL Server 2019 & WideWorldImporters sample database.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">As a Pre-requisites, </span><a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/install/sql-r-services-windows-install?view=sql-server-2017" style="background-color: white; color: #00749e; cursor: pointer; outline: none; text-decoration-line: none;" target="_blank">Install R services <img alt="Jump" aria-hidden="true" border="0" src="https://social.technet.microsoft.com/wiki/cfs-file.ashx/__key/communityserver-components-sitefiles/10_5F00_external.png" style="height: auto !important; max-width: 100%; overflow: hidden;" title="This link is external to TechNet Wiki. It will open in a new window." /> </a><span style="background-color: white;">and then from SSMS enable the external scripting feature. Restart the database engine and then verify the installation as mentioned in MSDN.</span></span><br />
<div style="background-color: white; color: #2a2a2a;">
<br /></div>
<div style="background-color: white; color: #2a2a2a;">
<span style="color: black;"><span style="font-family: Arial, Helvetica, sans-serif;">You might face “Access is denied” error while executing below scripts, to overcome that right-click on the folder where CSV files will be imported / exported and provide folder access (read, write) to "ALL_APPLICATION_PACKAGES".</span></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">For this approach, I have created a scalar function and Stored procedure.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Scalar function - returns a unique table name for a session. This function is required only if adhoc sql query has to be exported to csv file.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Stored procedure - Has below three input parameters and writes the output to csv file</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">i) </span><span style="font-family: "arial" , "helvetica" , sans-serif;">@ExportPath - path for exporting csv file</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">ii) @ObjectlisttoExport - list of tables, views to be exported</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">iii) @Objectlistdelim - delimiter for above object list</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This script will work from SQL Server 2017 and above. Actually it can be made to work from SQL Server 2016 & above - by modifying STRING_AGG that has been used </span><span style="font-family: "arial" , "helvetica" , sans-serif;">inside the stored procedure </span><span style="font-family: "arial" , "helvetica" , sans-serif;">to combine the list of columns into comma separated string</span><span style="font-family: "arial" , "helvetica" , sans-serif;">. This approach will not work below SQL Server 2016 version as execution of R language using T-SQL was introduced in SQL Server 2016.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>Scalar function:</i></span><br />
<br />
<pre class="brush:sql">USE [WideWorldImporters]
GO
CREATE OR ALTER FUNCTION fngetcsvtblname()
RETURNS NVARCHAR(128)
AS
BEGIN
DECLARE @tblname NVARCHAR(128)
SELECT @tblname = CONCAT('ExportCSV_',@@SPID,DB_ID(),'_temptbl')
RETURN @tblname
END;
GO
</pre>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i>Stored procedure:</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><br />
</i></span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><i></i> </span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><br />
<br />
<pre class="brush:sql">CREATE OR ALTER PROC usp_ExportCSV (@ExportPath NVARCHAR(MAX),
@ObjectlisttoExport NVARCHAR(MAX),
@Objectlistdelim CHAR(1)
)
AS
BEGIN
SET NOCOUNT ON;
IF ISNULL(@ExportPath,'') <> ''
BEGIN
SELECT @ExportPath = REPLACE(@ExportPath,'\','/')
DECLARE @tblname NVARCHAR(128)
SELECT @tblname= dbo.fngetcsvtblname()
IF EXISTS (SELECT 1 FROM sys.tables WHERE name = @tblname) OR (ISNULL(@ObjectlisttoExport,'') <> '' AND ISNULL(@Objectlistdelim,'') <> '')
BEGIN
DECLARE @TableColList TABLE (Cols NVARCHAR(MAX),Tbl NVARCHAR(128))
IF EXISTS (SELECT 1 FROM sys.tables WHERE name = @tblname)
BEGIN
INSERT @TableColList
SELECT CONCAT('TRY_CONVERT(','VARCHAR(MAX),',C.name,') AS ',QUOTENAME(C.NAME)) Columns -- To cover poor data type conversions
,CONCAT(S.name,'.',O.name) TableName
FROM sys.objects O
JOIN sys.schemas S
ON S.schema_id = O.schema_id
JOIN sys.columns C
ON O.object_id = C.object_id
JOIN sys.types TY
ON C.[user_type_id] = TY.[user_type_id]
WHERE CONCAT(S.name,'.',O.name) = (SELECT CONCAT(schema_name(schema_id),'.',name ) FROM sys.tables WHERE name = @tblname)
AND TY.name <> 'image' -- Ignore the datatypes that are not required to be exported
AND O.type IN ('U','V')
END
IF ISNULL(@ObjectlisttoExport,'') <> ''
BEGIN
INSERT @TableColList
SELECT CONCAT('TRY_CONVERT(','VARCHAR(MAX),',C.name,') AS ',QUOTENAME(C.NAME)) Columns -- To cover poor data type conversions
,CONCAT(S.name,'.',O.name) TableName
FROM sys.objects O
JOIN sys.schemas S
ON S.schema_id = O.schema_id
JOIN sys.columns C
ON O.object_id = C.object_id
JOIN sys.types TY
ON C.[user_type_id] = TY.[user_type_id]
WHERE CONCAT(S.name,'.',O.name) IN (SELECT value FROM STRING_SPLIT(@ObjectlisttoExport, @Objectlistdelim))
AND TY.name <> 'image' -- Ignore the datatypes that are not required to be exported
AND O.type IN ('U','V')
END
DROP TABLE IF EXISTS #ExportTablesList
SELECT IDENTITY(INT,1,1) Rn
,tbl
,STRING_AGG(cols,',') AS cols
INTO #ExportTablesList
FROM @TableColList
GROUP BY tbl
DECLARE @I INT = 1
,@SQL NVARCHAR(MAX) = N''
,@RScript NVARCHAR(MAX) = N''
,@filename NVARCHAR(MAX) = N''
WHILE @I <= (SELECT MAX(Rn) FROM #ExportTablesList)
BEGIN
--just for testing selecting top 10, this can be removed
SELECT @SQL = CONCAT('SELECT TOP 10',Cols,' FROM ',tbl,';')
,@tblname = Tbl
FROM #ExportTablesList WHERE Rn = @I
SELECT @tblname = REPLACE(@tblname,'.','_')
SELECT @filename = CASE WHEN SUBSTRING (@ExportPath,LEN(@ExportPath),1) = '/' THEN CONCAT(@ExportPath,@tblname,'.csv')
ELSE CONCAT(@ExportPath,'/',@tblname,'.csv') END
SET @RScript = CONCAT('write.csv(InputDataSet, file = "',@filename,'")')
EXEC sp_execute_external_script
@language = N'R'
,@script = @RScript
,@input_data_1 = @SQL
SET @tblname = ''
SET @filename = ''
SET @I = @I + 1
END
--After successful processing, dropping the table created for exporting adhoc sql into csv
SELECT @tblname= dbo.fngetcsvtblname()
IF EXISTS (SELECT 1 FROM sys.tables WHERE name = @tblname)
BEGIN
EXEC('DROP TABLE '+@tblname)
END
END ELSE PRINT 'No object specified for exporting to CSV or Objectlist params are empty'
END ELSE PRINT 'Export folder path need to be mentioned'
END;
GO
</pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Let us see the different execution results:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">1) When all the parameters are passed blank</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyNdM4jhhFSJwI4vvss-OT1Qt0-68h6hyphenhyphenTBlFpko_KRmAUnbfrAuJv690Jg1Kk6udQjGSncUbH6qAcTztY4jymYahSndowMZaq4iyV2TfYAIEheG-6CLoEtsEOFeEKKuX-C5v3t_vrAZ7l/s1600/T1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="455" data-original-width="802" height="362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyNdM4jhhFSJwI4vvss-OT1Qt0-68h6hyphenhyphenTBlFpko_KRmAUnbfrAuJv690Jg1Kk6udQjGSncUbH6qAcTztY4jymYahSndowMZaq4iyV2TfYAIEheG-6CLoEtsEOFeEKKuX-C5v3t_vrAZ7l/s640/T1.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">2) When folder path is passed but no objects specified for exporting</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuwOfUosa67SnDKEoNLO0PdXmARdCC5BiwybBdeOnoUYlvqoRL2AwGmzJMRVEGPhI3nEvvKYYUGwviMV8EWAxlGPpsFFIPR_O3ByuirNcg65TsLnhETn6_0G0-gr-g0IovhohlAo1a4vjP/s1600/T2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="457" data-original-width="806" height="362" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuwOfUosa67SnDKEoNLO0PdXmARdCC5BiwybBdeOnoUYlvqoRL2AwGmzJMRVEGPhI3nEvvKYYUGwviMV8EWAxlGPpsFFIPR_O3ByuirNcg65TsLnhETn6_0G0-gr-g0IovhohlAo1a4vjP/s640/T2.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">3) Notice in the below screenshot, I am getting a unique table name using scalar function and inserting my </span><span style="font-family: "arial" , "helvetica" , sans-serif;">adhoc sql output into that table and then calling the stored procedure.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">This stored procedure exports the adhoc sql output to csv file in the mentioned path and drops the table created after successful processing.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKS0SS4KlM10Ch8kOcLiEZ0amNcXelXVWnqgSh-QyoBZePgVUaIG0iVo-lu_7SQJEtXEh9BGq2A-KpWv58j_lsKFLiXWyEWjEPJy98hdgE2D3mhoR_k78YWKBw786tBw0TIToGph8IeanC/s1600/T3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="563" data-original-width="802" height="448" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKS0SS4KlM10Ch8kOcLiEZ0amNcXelXVWnqgSh-QyoBZePgVUaIG0iVo-lu_7SQJEtXEh9BGq2A-KpWv58j_lsKFLiXWyEWjEPJy98hdgE2D3mhoR_k78YWKBw786tBw0TIToGph8IeanC/s640/T3.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">4) I am passing the list of tables from Sales schema in WideWorldImporters database.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This stored procedure exports the </span><span style="font-family: "arial" , "helvetica" , sans-serif;">list of tables data</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> into separate csv files with table name as file name in the mentioned path.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihOK8wYUuFcDn-cOTjwpM7nr90Y9NMD26vszGIvX9QgcILy00WP5AvGQ58ZcpzihmV3gtqpcoefrg0wQrTO1B-UzY1A5krFpimtNU9ZKPIqVPCK75MA_wgvEPIM2yx3SUacwVG2HszSRTA/s1600/T4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="157" data-original-width="812" height="122" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihOK8wYUuFcDn-cOTjwpM7nr90Y9NMD26vszGIvX9QgcILy00WP5AvGQ58ZcpzihmV3gtqpcoefrg0wQrTO1B-UzY1A5krFpimtNU9ZKPIqVPCK75MA_wgvEPIM2yx3SUacwVG2HszSRTA/s640/T4.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">5) This is to show that Stored procedure can export adhoc sql output, list of tables, views data into separate csv files simultaneously.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPG3WbCLO3nWzlOmCmEQGc-kCWi5psjeaAahsY-Ttmfqn6qoQ6aaGBl73YbnBU4gr9hnhj7THRLaqlAe2b0qTF4YUQDdmLovUQDp9G-uMKDs98Mn2TKo4fvK0VfDzdzlJkGIUW7w98cAmS/s1600/T5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="422" data-original-width="788" height="342" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPG3WbCLO3nWzlOmCmEQGc-kCWi5psjeaAahsY-Ttmfqn6qoQ6aaGBl73YbnBU4gr9hnhj7THRLaqlAe2b0qTF4YUQDdmLovUQDp9G-uMKDs98Mn2TKo4fvK0VfDzdzlJkGIUW7w98cAmS/s640/T5.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i>Sample execution code block:</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<pre class="brush:sql">SELECT dbo.fngetcsvtblname() --function to get unique table name for inserting adhoc sql into a table
DECLARE @tblname NVARCHAR(128) = dbo.fngetcsvtblname()
IF EXISTS (SELECT 1 FROM sys.tables WHERE name = @tblname)
BEGIN
EXEC('DROP TABLE '+@tblname)
END
/** place your adhoc sql to be exported into csv and SELECT INTO tablename provided by above function **/
SET NOCOUNT ON;
SELECT TOP 5 OL.OrderLineID,O.*
INTO ExportCSV_726_temptbl
FROM [WideWorldImporters].[Sales].[Orders] O
JOIN [Sales].[OrderLines] OL
ON O.OrderID = OL.OrderID
/** place your adhoc sql to be exported into csv and SELECT INTO tablename provided by above function **/
EXEC usp_ExportCSV 'H:\Temp\Blog\ImportExportExcel','SAles.Orders|Sales.Invoices|Website.Customers','|'
</pre>
<br />
<br />
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">See Also:</span></span></h3>
<ul style="text-align: left;">
<li><i style="font-family: arial, helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_59.html" style="font-family: arial, helvetica, sans-serif;" target="_blank">SQL Server Export/Import excel using R script</a></i></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://social.technet.microsoft.com/wiki/contents/articles/52484.sql-server-importexport-to-csv-using-r-script.aspx" target="_blank">My post on Technet wiki - SQL Server Import/Export to CSV using R script</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-58264224187330661302018-12-31T12:00:00.000+05:302019-07-06T01:53:00.566+05:30SQL Server Export/Import excel using R script<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see new approach to import excel into SQL Server and export SQL server data to excel. There are some existing methods to do this using BCP, Bulk Insert, Import & Export wizard from SSMS, SSIS, Azure data factory, Linked server & OPENROWSET query and SQLCMD.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">BULK INSERT statement, the BCP tool, or Azure Data Factory <a href="https://docs.microsoft.com/en-us/sql/relational-databases/import-export/import-data-from-excel-to-sql?view=sql-server-2017" target="_blank">can't read Excel files directly</a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">BCP - Work around has to be done to include the header </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">SSIS - With dynamic source & destination, handling mapping increases the complexity of the package</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">SQLCMD - Cannot export output in Excel</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">R & Python language extension was introduced in SQL Server 2016 & 2017 as part of machine learning. With support of <a href="https://blog.revolutionanalytics.com/2018/11/r-support-in-azure-sql-database.html" target="_blank">R in Azure SQL database</a> and <span style="background-color: white;"><a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/java/extension-java?view=sqlallproducts-allversions" target="_blank">Java language extension support in SQL Server 2019</a>, this new approach can be used extensively as it easy, fast and flexible.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white;"></span></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below series of posts has some examples on how to install various R packages (xlsx - read/write excel </span><span style="font-family: "arial" , "helvetica" , sans-serif;">& readxl - used for reading excel). </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Also I have showed examples on: </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">i) how to export SQL Server data into excel file with multiple tabs with table names as sheet names</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">ii) how to export SQL Server data into multiple excel files with table name as file & sheet names</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">iii) how to import excel with multiple sheets into different SQL Server table by creating target schema dynamically with sheet name as table name</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">iv) how to import multiple excel files in a directory (including sub folders) with multiple sheets </span><span style="font-family: "arial" , "helvetica" , sans-serif;">into different SQL Server table by creating target schema dynamically with file name + sheet name as table name</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">v) how to export output of SQL files stored in a folder to excel files</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">SQL Server Export/Import excel using R script - Part 1</a></i><br />
<i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">SQL Server Export/Import excel using R script - Part 2</a></i><br />
<i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">SQL Server Export/Import excel using R script - Part 3</a></i><br />
<i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_25.html" target="_blank">SQL Server Export/Import excel using R script - Part 4</a></i><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_41.html" target="_blank">SQL Server Export/Import excel using R script - Part 5</a></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_31.html" target="_blank"><span style="font-family: "arial" , "helvetica" , sans-serif;"></span>SQL Server Export/Import excel using R script - Part 6</a></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">Above examples are done using the base R packages & functions and very few are explicitly installed </span><span style="font-family: "arial" , "helvetica" , sans-serif;">to achieve some excel operations.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">With respect to moving data (Import / Export) between SQL Server & Excel, there are various scenarios based on each requirement. I have covered some of them but tweaking the solutions mentioned above can cover any scenario.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I'll keep updating this article with different examples. If anyone has specific requirement or any issues trying above examples, please comment.</span><br />
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">See Also:</span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2019/01/sql-server-export-adhoc-sql-or-table-to.html" target="_blank">SQL Server export adhoc SQL or table to CSV file</a></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://www.allaboutmssql.com/2019/01/import-csv-or-text-file-into-sql-server.html" target="_blank"><i>Import csv or text file into SQL Server table</i></a></span><br />
<br /></div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-36247203617539888282018-12-31T00:08:00.001+05:302019-07-06T01:50:59.892+05:30SQL Server Export/Import excel using R script - Part 6<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In continuation to my previous posts on SQL Server Export/Import excel using R script <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">Part 1</a>, <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">Part 2</a>, <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">Part 3</a>, <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_25.html" target="_blank">Part 4</a> & <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_41.html" target="_blank">Part 5</a>. In this post, let us see how to invoke sql files in a folder and export the output of the query as Excel files.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As a prerequisite, we need R packages "xlsx" & "rJava", which I had showed in my previous posts on how to install and load them.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As we are going to make use of the powershell SQL Server cmdlets "Invoke-Sqlcmd", </span><span style="font-family: "arial" , "helvetica" , sans-serif;">let us make sure <a href="https://docs.microsoft.com/en-us/sql/powershell/download-sql-server-ps-module?view=sql-server-2017" target="_blank"><i>SQL Server powershell modules are downloaded & installed from Powershell</i></a>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For this example, I have created below folder structure:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ_tst3vZTaoEW8-7q8Edr0oQI3SbP2V1ma4hMSmn7Bzw57P1AxEAp6y7OpYX3U80KwwAB6wITpcpBToE8icE4smJ9Eckd37LHxn2GhQnKHIqSqAP32piXxMY1CO1XyP05j8L1AyUCP-m5/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="214" data-original-width="556" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ_tst3vZTaoEW8-7q8Edr0oQI3SbP2V1ma4hMSmn7Bzw57P1AxEAp6y7OpYX3U80KwwAB6wITpcpBToE8icE4smJ9Eckd37LHxn2GhQnKHIqSqAP32piXxMY1CO1XyP05j8L1AyUCP-m5/s640/1.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">Temp folder - Is a placeholder for staging files created in the intermediate process</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Script folder - Has the R script that runs this entire process</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">BusinessReports folder - Where excel reports are exported</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">SQL folder - Contains the SQL files to be executed</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For this example, I have placed three .sql files in a folder and the sql inside the file are as shown in below screenshot. All three sql queries are selecting data from three different tables in "WideWorldImporters" sample database.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOm2B_C70OvqBxf6ImSUebYGqjek76-TAya00fna0TK07OEXjELhNWzrJ-PUCx2abfYsKRwIGadW5smIIQ_FmznphHSPCYsLamZzTDAw0JJB-6GZ_2qG-wBhsj6rGomTumjaCvvIB8TNSc/s1600/2.JPG" imageanchor="1" style="font-family: "Times New Roman"; margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="212" data-original-width="655" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOm2B_C70OvqBxf6ImSUebYGqjek76-TAya00fna0TK07OEXjELhNWzrJ-PUCx2abfYsKRwIGadW5smIIQ_FmznphHSPCYsLamZzTDAw0JJB-6GZ_2qG-wBhsj6rGomTumjaCvvIB8TNSc/s640/2.JPG" width="640" /></a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOFDI9aQ3vTId92vGoli_zwUpXcm90dIG1pl2k16dSRf9RDZNyXwyxCgasbx0ebYL-ADRBOHS-KLwrosyzHOobV7pgpfzWRcaM3oRRV01HDbVog6L107RSv_7nQKr18A9Bv0-3RTvGKsw8/s1600/3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="596" data-original-width="1054" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOFDI9aQ3vTId92vGoli_zwUpXcm90dIG1pl2k16dSRf9RDZNyXwyxCgasbx0ebYL-ADRBOHS-KLwrosyzHOobV7pgpfzWRcaM3oRRV01HDbVog6L107RSv_7nQKr18A9Bv0-3RTvGKsw8/s640/3.JPG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u><i>Explanation on how this approach works:</i></u></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">R invokes the powershell command using "system2" function. Powershell command then reads all the sql files (using Invoke-Sqlcmd</span><span style="font-family: "arial" , "helvetica" , sans-serif;">) and writes the sql result to a text file (just as a staging)</span><span style="font-family: "arial" , "helvetica" , sans-serif;">. Only one staging file will be created for entire process of exporting SQL files output to Excel and will be </span><span style="font-family: "arial" , "helvetica" , sans-serif;">deleted at the end of the process. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Using the data in staging text file on each iteration, an excel is created using "xlsx" r package.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">Below R script does everything what I have mentioned in above explanation:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOeomLeNAMy6iOSqZsNwVlLGTuwYrV8Z1hPXKyk4dzfFgkrG5Ov-4lWxXs0FzF5PXDeMEOkIEUQAPkEo10kQIEG5P6C9kX1TCl4icPFXpnlVMFcPgGX7_AgD4dkPoUgiDx2SopSGLweDEl/s1600/4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="673" data-original-width="986" height="436" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOeomLeNAMy6iOSqZsNwVlLGTuwYrV8Z1hPXKyk4dzfFgkrG5Ov-4lWxXs0FzF5PXDeMEOkIEUQAPkEo10kQIEG5P6C9kX1TCl4icPFXpnlVMFcPgGX7_AgD4dkPoUgiDx2SopSGLweDEl/s640/4.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have saved the above script in Script folder, it can be either manually opened & executed from any R environment (R studio etc.) or this script can be invoked from command line.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdXgWUSfQ9_c_JZAdcHzlBdyJiQXW-WuuzvyJXqFoog5jBzhs4WN0Fn2yL7WiDj63VLFNFo395eh7ImZJMYoed_1_xDDxlt1Ql_X66UGNYb4lXhY7_5SUTJRixegyWbdbH2yANBaPlcTWo/s1600/5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="171" data-original-width="619" height="176" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgdXgWUSfQ9_c_JZAdcHzlBdyJiQXW-WuuzvyJXqFoog5jBzhs4WN0Fn2yL7WiDj63VLFNFo395eh7ImZJMYoed_1_xDDxlt1Ql_X66UGNYb4lXhY7_5SUTJRixegyWbdbH2yANBaPlcTWo/s640/5.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below screenshot shows on how to invoke R script from command Prompt </span><span style="font-family: "arial" , "helvetica" , sans-serif;">by using Rscript.exe in</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> SQL Server folder</span><span style="font-family: "arial" , "helvetica" , sans-serif;">:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Goto Run -> cmd -> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj07Rm9Ln0xAh_Bq8IRqnfJIBkAvaxVblzGbySStEt0iLs-Wm4PhCSz0XzsGdF62bL3mj9RXgNKDVUrPY3TgPtj-kmA-x0u72vyQcmn-4qRB2YU-BJuPojml5sX9VVs6W0rR07KsctBOVlw/s1600/6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="513" data-original-width="980" height="334" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj07Rm9Ln0xAh_Bq8IRqnfJIBkAvaxVblzGbySStEt0iLs-Wm4PhCSz0XzsGdF62bL3mj9RXgNKDVUrPY3TgPtj-kmA-x0u72vyQcmn-4qRB2YU-BJuPojml5sX9VVs6W0rR07KsctBOVlw/s640/6.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This command line can also be created as a SQL agent job and scheduled as per requirement.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: black; color: white; font-family: "arial" , "helvetica" , sans-serif;">CD "C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\R_SERVICES\bin"</span><br />
<span style="background-color: black; color: white; font-family: "arial" , "helvetica" , sans-serif;">Rscript.exe "H:\Temp\Blog\ImportExportExcel\Script\SQLExportExcel.R"</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">After executing the R script from Command Prompt or R studio:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can see the output of SQL files in Excel format within a directory created with date and Excel file & sheet name is SQL file name</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF5k5ABdv-MpDWBtMg4pCUSVE3N7hVUA2ZXzR5i3sQWGKaGNPMBIpJZ5fumSgtg-QSdx0CTgyJYzIV4-cpRl6Vaq_g3QZa2YiGIfpki8SyM9q9MhumzCgmJDItwf9NqFBozht2QcbhyphenhyphenI0r/s1600/7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="465" data-original-width="774" height="384" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhF5k5ABdv-MpDWBtMg4pCUSVE3N7hVUA2ZXzR5i3sQWGKaGNPMBIpJZ5fumSgtg-QSdx0CTgyJYzIV4-cpRl6Vaq_g3QZa2YiGIfpki8SyM9q9MhumzCgmJDItwf9NqFBozht2QcbhyphenhyphenI0r/s640/7.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Executing the R script on same day, just overwrites the directory & excel files without error and a separate directory is created when the script is executed on the next day.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I tested this scenario with the above script, where if one of the SQL file has incorrect syntax, script still works for all the other SQL files and runs the entire process. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Using password argument in write.xlsx2 function, the exported excel files can be password protected. But unfortunately writing of password protected workbooks is supported for Excel 2007 OOXML format only.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below is the complete code block:</span><br />
<span style="font-family: arial, helvetica, sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">Please note: In the below script, we need to configure the values for </span><br />
<span style="font-family: arial, helvetica, sans-serif;">database connection details and </span><span style="font-family: arial, helvetica, sans-serif;">folder path details </span><br />
<div>
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">library(foreach);</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">library(rJava);</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">library("xlsx");</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">#getting list of sql files & file names from folder</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">list_filespath <- list.files(path = "H:/Temp/Blog/ImportExportExcel/SQL", pattern = ".sql", all.files = FALSE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> full.names = TRUE, recursive = FALSE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> ignore.case = FALSE, include.dirs = FALSE, no.. = TRUE)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">list_filenames <- list.files(path = "H:/Temp/Blog/ImportExportExcel/SQL", pattern = ".sql", all.files = FALSE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> full.names = FALSE, recursive = FALSE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> ignore.case = FALSE, include.dirs = FALSE, no.. = TRUE)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">#Check file exists in path</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">if (length(list_filespath) != 0) {</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> #creating new directory on each day for SQL output </span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> newdir <- paste("H:/Temp/Blog/ImportExportExcel/BusinessReports/Report", Sys.Date(), sep = "_")</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> dir.create(newdir)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> foreach(m = 1:length(list_filespath)) %do%</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> {</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> #forming the powershell command to invoke sql files </span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> command1 <- "Invoke-Sqlcmd -ServerInstance 'LAPTOP-LCUB6HBB' -Database WideWorldImporters -InputFile "</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> command3 <- "| Export-Csv 'H:/Temp/Blog/ImportExportExcel/Temp/Stagingfile.txt'"<span style="white-space: pre;"> </span></span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> command <- paste(command1,list_filespath[m],command3, sep=" ")</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> system2("powershell", args = command)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> #powershell Invoke-sqlcmd output to staging file in .txt format</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> StagingOut <- read.table('H:/Temp/Blog/ImportExportExcel/Temp/Stagingfile.txt', skip = 1, header =TRUE, sep =',')</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> #create excel output using .txt staging file</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> ExcelFilename <- paste(newdir,"/",gsub(".sql", "", list_filenames[m]),".xlsx", sep="")</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> write.xlsx2(StagingOut, file = ExcelFilename, sheetName=list_filenames[m],</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> col.names=TRUE, append=FALSE, password= "Password123")</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> }</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">}</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">#Delete staging file (intermediate file before exporting to excel) if it exist</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">if (file.exists('H:/Temp/Blog/ImportExportExcel/Temp/Stagingfile.txt')) </span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">{</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> file.remove('H:/Temp/Blog/ImportExportExcel/Temp/Stagingfile.txt')</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">}</span></span><br />
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>See Also:</i></span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_59.html" target="_blank">SQL Server Export/Import excel using R script</a></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://social.technet.microsoft.com/wiki/contents/articles/52624.sql-server-export-sql-file-output-to-excel-file.aspx" target="_blank">My post on Technet wiki - SQL Server - Export SQL file output to Excel file</a></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div style="text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: x-small;"><i>Happy new year 2019 !!</i></span></div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-47816296396860517982018-12-26T04:13:00.002+05:302019-07-06T01:45:23.904+05:30SQL Server Export/Import excel using R script - Part 5<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In continuation to my previous posts on SQL Server Export/Import excel using R script <i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">Part 1</a>, <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">Part 2</a>, <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">Part 3</a> & <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_25.html" target="_blank">Part 4</a></i>. In this post, let us see how to import multiple excel files in a folder (including sub folders) into different SQL Server table created dynamically based on excel file structure.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For this example, I am going to make use of excel files generated using second script in <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank"><i>Part 2</i></a> post.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Also I have created sub folder named "Copy" and copied the same set of files into it, I have done this to ensure script has traversed through all the folders and works fine.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Also to test if the script can ignore the empty sheets, I have manually opened 1 or 2 excel files added an empty sheet and saved it.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this example, SQL Server table name is made as file name + sheet name and removed all special characters and spaces if any from file/sheet name to make sure SQL Server table name is unique and as per MSSQL standard.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu5y_S1uvAM5Ji4AhHSFtfmRG1tHs6HiZPLodAHR7a-sJJnT8_TrQESnUHW2V_SZ-pzH1jZJP_P7luDW-HOb7Sb5Za6PSDCKkKHxPNj8bjgrX6Y6Kh0jkmu3t7cSQI7zTj7Jh6imbpWEKX/s1600/P1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="282" data-original-width="616" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu5y_S1uvAM5Ji4AhHSFtfmRG1tHs6HiZPLodAHR7a-sJJnT8_TrQESnUHW2V_SZ-pzH1jZJP_P7luDW-HOb7Sb5Za6PSDCKkKHxPNj8bjgrX6Y6Kh0jkmu3t7cSQI7zTj7Jh6imbpWEKX/s640/P1.JPG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Q5nWjxpyUaXsBzXWXPWC78ebFKhH1oARQ5aeQ9nDmtDQN3zqIRaFyoZlsKK_Fa7TDEYt46I8XqRnvgIkbdsruBL7F_EokTge7VVv_y1YoIFR7CM3YSBysQ9R1aHrhbIufFy70eAI_Lx5/s1600/P3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="625" data-original-width="1143" height="348" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Q5nWjxpyUaXsBzXWXPWC78ebFKhH1oARQ5aeQ9nDmtDQN3zqIRaFyoZlsKK_Fa7TDEYt46I8XqRnvgIkbdsruBL7F_EokTge7VVv_y1YoIFR7CM3YSBysQ9R1aHrhbIufFy70eAI_Lx5/s640/P3.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As you can see in above screenshot, table names with Copy as prefix are imported from sub folders.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below is the script that iterates through <u><b>multiple files with multiple sheets and create them as separate table in SQL Server</b></u>:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">Please note: In the below script, we need to configure the values for </span><br />
<span style="font-family: arial, helvetica, sans-serif;">sqlConnString variable with database connection details</span><br />
<span style="font-family: arial, helvetica, sans-serif;">list_filespath & list_filenames variable with folder path details </span><br />
<div>
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">DECLARE @RScript NVARCHAR(MAX) =N' </span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> library(readxl);</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> library(foreach);</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> sqlConnString <- "Driver=SQL Server;Server=''LAPTOP-LCUB6HBB''; Database=Staging;Uid=sa;Pwd=***"</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> list_filespath <- list.files(path = "H:/Temp/Blog/ImportExportExcel", pattern = "*.xlsx", all.files = FALSE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> full.names = TRUE, recursive = TRUE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> ignore.case = FALSE, include.dirs = FALSE, no.. = TRUE)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> list_filenames <- list.files(path = "H:/Temp/Blog/ImportExportExcel", pattern = "*.xlsx", all.files = FALSE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> full.names = FALSE, recursive = TRUE,</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> ignore.case = FALSE, include.dirs = FALSE, no.. = TRUE)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><span style="white-space: pre;"> </span>#Check file exists in path</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> if (length(list_filespath) != 0) {</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> foreach(m = 1:length(list_filespath)) %do%</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> {</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> filepath_var <- list_filespath[m]</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> tab_names <- excel_sheets(path = filepath_var)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> foreach(n = 1:length(tab_names)) %do%</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> {</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> sqlTable <- paste(strsplit(list_filenames[m],split = ".xlsx"),tab_names[n],sep = "_") </span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> sqlTable <- gsub("\\s", "", sqlTable)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> sqlTable <- gsub("[^[:alnum:]]", "", sqlTable)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> Output <- read_excel(path = filepath_var, sheet = n)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> #check if the sheet has header </span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> if (dim(Output)[2] != 0) {</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> sqlDS <- RxSqlServerData(connectionString = sqlConnString,table = sqlTable)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> rxDataStep(inData = Output, outFile = sqlDS,overwrite = TRUE)</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> }</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> }</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> }</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> }'</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"><br /></span></span>
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;">EXEC sp_execute_external_script</span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> @language = N'R'</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"></span></span><br />
<span style="color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #3d85c6;"> ,@script = @RScript</span></span><br />
<div>
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">More information on this topic, to be continued in <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_31.html" target="_blank"><i>part 6</i></a> ..</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">See Also:</span></h3>
<ul style="text-align: left;">
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">SQL Server Export/Import excel using R script - Part 1</a></i></li>
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">SQL Server Export/Import excel using R script - Part 2</a></i></li>
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">SQL Server Export/Import excel using R script - Part 3</a></i></li>
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_25.html" target="_blank">SQL Server Export/Import excel using R script - Part 4</a></i></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://social.technet.microsoft.com/wiki/contents/articles/52528.sql-server-importexport-to-excel-using-r-script.aspx" target="_blank">My post on Technet Wiki - SQL Server Import/Export to Excel using R script</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-41550811370524433912018-12-25T22:12:00.001+05:302019-07-06T01:42:22.254+05:30SQL Server Export/Import excel using R script - Part 4<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span data-iceapw="7" style="background-color: white;">In continuation to my previous posts on </span><span data-iceapw="10" style="background-color: white;"><i>SQL Server Export/Import excel using R script - <span style="color: #0b5394;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">Part 1</a></span> ,<span style="color: #0b5394;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">Part 2</a></span> & <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">Part3</a></i></span><span style="background-color: white;"> where we saw about exporting to Excel and importing to SQL Server table using "xlsx" package which is a powerful package but it has external dependency on Java ("rJava" package).</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white;"><br />
</span></span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white;">In this post, let us see how to import excel with multiple sheets using "readxl" package into different SQL Server tables.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white;"><br />
</span></span> <br />
<div class="MsoNormal" data-iceapc="2" data-iceapw="17" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="1" data-iceapw="17"><span style="font-family: "arial" , "helvetica" , sans-serif;">I have done below example using <i><a href="http://www.allaboutmssql.com/2018/12/sql-server-2019-installation.html" target="_blank">SQL Server 2019</a></i> on windows 10 (64 bit) and R (3.4.4).<o:p></o:p></span></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;">Refer my previous post on how to <i data-iceapc="1"><span style="color: blue;"><a href="http://www.allaboutmssql.com/2017/02/sql-server-2016-getting-started-with-r.html" target="_blank">install R services</a></span></i>. </span></span><br />
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;">I have also used Wideworldimporters sample database for data setup in this example. </span></span><br />
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;">Before trying to </span></span><span style="font-family: "arial" , "helvetica" , sans-serif;">import excel with multiple sheets using "readxl" package into SQL Server tables, I tried to re-create the Excel file that was created using script provided in <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank"><i>part 2</i></a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">I got below error :</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-B_loKHOWpF63Ju79y70OFV_vL-9Vkt2_HLSsuWgnZGt-4pMuisWuEmfzLnfN8dpq_r7Khb0AyKOiM-QGfPQLH5J-zmGOvf5Cf3qWLDcuXHZ7-oUGTbBJC9FWXxnIJ1oLMwzGWkveZzGn/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="595" data-original-width="1033" height="368" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-B_loKHOWpF63Ju79y70OFV_vL-9Vkt2_HLSsuWgnZGt-4pMuisWuEmfzLnfN8dpq_r7Khb0AyKOiM-QGfPQLH5J-zmGOvf5Cf3qWLDcuXHZ7-oUGTbBJC9FWXxnIJ1oLMwzGWkveZzGn/s640/1.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span style="color: red; font-family: "courier new" , "courier" , monospace;"><b>Error in .jnew("java/io/FileOutputStream", jFile) : </b></span><br />
<span data-iceapc="3" data-iceapw="8"><b><span style="color: red; font-family: "courier new" , "courier" , monospace;"></span></b></span><br />
<span style="color: red; font-family: "courier new" , "courier" , monospace;"><b> java.io.FileNotFoundException: (Access is denied)</b></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;">This is because of Java language extension support in SQL Server 2019, So I had to provide folder access (read, write) to "ALL_APPLICATION_PACKAGES".</span></span><br />
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwBHTyjZS_DT8G-XSn9s_vm0-nN7e0CIR7rc5S24qWYdrFU220T-rsN_mb7_PCkE0eZNKPQAG1agFKfy4KeIa5Zkp6v22aCn6ZzBNs9eXn1c0BbeG1peBDsKoE4MhyhlM3A-N_Gdp_b_3u/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="484" data-original-width="960" height="322" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwBHTyjZS_DT8G-XSn9s_vm0-nN7e0CIR7rc5S24qWYdrFU220T-rsN_mb7_PCkE0eZNKPQAG1agFKfy4KeIa5Zkp6v22aCn6ZzBNs9eXn1c0BbeG1peBDsKoE4MhyhlM3A-N_Gdp_b_3u/s640/2.JPG" width="640" /></a></div>
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 1:</b></span></span><br />
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;">Now let us install the required packages for this example:</span></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="3" data-iceapw="8"><span style="font-family: "arial" , "helvetica" , sans-serif;">Open R.exe from below path:</span></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="line-height: normal; margin-bottom: 0cm;">
<span style="background-color: black; color: white; font-family: "arial" , "helvetica" , sans-serif;">C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\R_SERVICES\bin</span><br />
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
</div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="line-height: normal; margin-bottom: 0cm;">
<span data-iceapc="3" data-iceapw="8" style="background-color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">From R.exe, execute </span></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #0b5394; color: white;">install.packages("readxl")</span> </span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: transparent;"><b>Step 2:</b></span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: transparent;"><br />
</span></span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: transparent;">After successful installation of above package, </span>Copy & paste all the packages downloaded from above step into default R library to SQL Server R_services library.</span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="line-height: normal; margin-bottom: 0cm;">
<span style="background-color: white; font-family: "arial" , "helvetica" , sans-serif;">From </span><span style="background-color: black; color: white; font-family: "arial" , "helvetica" , sans-serif;">C:\Users\Sathy\Documents\R\win-library\3.4</span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="line-height: normal; margin-bottom: 0cm;">
<span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;">to </span></span><span style="background-color: black; color: white; font-family: "arial" , "helvetica" , sans-serif;">C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\R_SERVICES\library</span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Step 3:</b></span></span><br />
<span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Below script will import Excel file with six sheets into six different table on database called "Staging" that I have created before executing this script</span></span><br />
<span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span> <span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span data-iceapw="4" style="font-size: 15.4px;">Sheet 1 - CustomerTransactions</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;" /><span data-iceapw="4" style="font-size: 15.4px;">Sheet 2 - InvoiceLines</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;" /><span data-iceapw="4" style="font-size: 15.4px;">Sheet 3 - Invoices</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;" /><span data-iceapw="4" style="font-size: 15.4px;">Sheet 4 - OrderLines</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;" /><span data-iceapw="4" style="font-size: 15.4px;">Sheet 5 - Orders</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px;" /><span data-iceapw="4" style="font-size: 15.4px;">Sheet 6 - SpecialDeals</span></span></span><br />
<span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><span data-iceapw="4" style="font-size: 15.4px;"><br /></span></span></span>
<span style="font-family: arial, helvetica, sans-serif;"><span style="font-size: 15.4px;">Please note: In the below script, we need to configure the values for </span></span><br />
<span style="font-family: arial, helvetica, sans-serif;"><span style="font-size: 15.4px;">sqlConnString variable with database connection details</span></span><br />
<span style="background-color: transparent;"><span style="font-size: 15.4px;"><span data-iceapw="4" style="font-family: arial, helvetica, sans-serif;"></span></span></span><br />
<span style="font-family: arial, helvetica, sans-serif;"><span style="font-size: 15.4px;">filepath_var variable with folder path details</span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="line-height: normal; margin-bottom: 0cm;">
<div style="background-color: white;">
<br /></div>
<div style="background-color: white;">
<br /></div>
<span style="background-color: #3d85c6; color: white;">DECLARE @RScript NVARCHAR(MAX) =N' </span><br />
<span style="background-color: #3d85c6; color: white;"> library(readxl);</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>library(foreach);</span><br />
<span style="background-color: #3d85c6; color: white;"> filepath_var <- "H:/Temp/Blog/ImportExportExcel/Sales.xlsx"</span><br />
<span style="background-color: #3d85c6; color: white;"> tab_names <- excel_sheets(path = filepath_var)</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>sqlConnString <- "Driver=SQL Server;Server=''LAPTOP-LCUB6HBB''; Database=Staging;Uid=sa;Pwd=***"</span><br />
<span style="background-color: #3d85c6; color: white;"><br />
</span> <span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>foreach(n = 1:length(tab_names)) %do%</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>{</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>sqlTable <- tab_names[n]</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>Output <- read_excel(path = filepath_var, sheet = n)</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>sqlDS <- RxSqlServerData(connectionString = sqlConnString,table = sqlTable)</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>rxDataStep(inData = Output, outFile = sqlDS,overwrite = TRUE)</span><br />
<span style="background-color: #3d85c6; color: white;"><span style="white-space: pre;"> </span>}'</span><br />
<span style="background-color: #3d85c6; color: white;"><br />
</span> <span style="background-color: #3d85c6; color: white;">EXEC sp_execute_external_script</span><br />
<span style="background-color: #3d85c6; color: white;"> @language = N'R'</span><br />
<span style="background-color: #3d85c6; color: white;"> ,@script = @RScript</span><br />
<div style="background-color: white;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhyphenhyphenMRxwkW_wgkbhryjEMVM3N6FIPGCu5TdCXr8s87hYADBeXkQwk7W7T-g0Bnk6CAnteAGGgIc514Vk6NW8ix9fb44b2rGQGKnfb1uW4U_nSENjOOgClkW-prN13IZPkpnolHynlxr15Rh/s1600/3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="622" data-original-width="1299" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhyphenhyphenMRxwkW_wgkbhryjEMVM3N6FIPGCu5TdCXr8s87hYADBeXkQwk7W7T-g0Bnk6CAnteAGGgIc514Vk6NW8ix9fb44b2rGQGKnfb1uW4U_nSENjOOgClkW-prN13IZPkpnolHynlxr15Rh/s640/3.JPG" width="640" /></a></div>
<div style="background-color: white;">
<br /></div>
</div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<span style="background-color: transparent;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span></div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="line-height: normal; margin-bottom: 0cm;">
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">I have updated above script to make sure script doesn't error out if the file does not exist in the path or if any of the sheet is empty. Below is the updated script:</span></div>
<div style="background-color: white;">
<br /></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">To test the script I have opened the "Sales.xlsx" file and added an empty sheet and saved it.</span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div class="separator" style="background-color: white; clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgx-p9d_ayjJrp9G-6oMzOjpU79e3Qvzo0C7mhpeCvZ7ih2AAm7E4dUU_kZrj7H5VJjU-qCzCWq2j_-z4PeWQG9P1rKovvueXwxkZD7KmKx252EoR9Gy03V7oqwQpiJCxRmRtwbis87w0vp/s1600/4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="627" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgx-p9d_ayjJrp9G-6oMzOjpU79e3Qvzo0C7mhpeCvZ7ih2AAm7E4dUU_kZrj7H5VJjU-qCzCWq2j_-z4PeWQG9P1rKovvueXwxkZD7KmKx252EoR9Gy03V7oqwQpiJCxRmRtwbis87w0vp/s640/4.JPG" width="598" /></a></div>
<div class="separator" style="background-color: white; clear: both; text-align: center;">
<br /></div>
<div class="separator" style="background-color: white; clear: both; text-align: left;">
<br /></div>
<div class="separator" style="background-color: white; clear: both;">
<span style="font-family: Arial, Helvetica, sans-serif;">Please note: In the below script, we need to configure the values for </span></div>
<div class="separator" style="background-color: white; clear: both;">
<span style="font-family: Arial, Helvetica, sans-serif;">sqlConnString variable with database connection details</span></div>
<div class="separator" style="background-color: white; clear: both;">
<span style="font-family: Arial, Helvetica, sans-serif;">filepath_var variable with folder path details</span></div>
<div style="background-color: white;">
<br /></div>
<div style="background-color: white;">
<br /></div>
<span style="background-color: #3d85c6;"><span style="color: white;">DECLARE @RScript NVARCHAR(MAX) =N' </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> library(readxl);</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> library(foreach);</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>filepath_var <- "H:/Temp/Blog/ImportExportExcel/Sales.xlsx"</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>sqlConnString <- "Driver=SQL Server;Server=''LAPTOP-LCUB6HBB''; Database=Staging;Uid=sa;Pwd=***"</span></span><br />
<span style="background-color: #3d85c6; white-space: pre;"><span style="color: white;"> </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>#Check file exists in path</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>if (file.exists(filepath_var)) {</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br /></span></span>
<span style="background-color: #3d85c6;"><span style="color: white;"> tab_names <- excel_sheets(path = filepath_var)</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> foreach(n = 1:length(tab_names)) %do%</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> {</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span> sqlTable <- tab_names[n]</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span> Output <- read_excel(path = filepath_var, sheet = n)</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> #check if the sheet has header </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> if (dim(Output)[2] != 0) {</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br /></span></span>
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span> sqlDS <- RxSqlServerData(connectionString = sqlConnString,table = sqlTable)</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span> rxDataStep(inData = Output, outFile = sqlDS,overwrite = TRUE)</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> }</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> }</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> }'</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br /></span></span>
<span style="background-color: #3d85c6;"><span style="color: white;">EXEC sp_execute_external_script</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> @language = N'R'</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> ,@script = @RScript</span></span><br />
<div style="background-color: white;">
<br /></div>
<div class="separator" style="background-color: white; clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisloVqGopJpTGeEqTqJs-OW8uZzu-_8DXa5RHPwSTmKcJJVs12GsWyu693YVAMeZTqhhRKKYRy7oompxGxl9rpRKRNrL6Qqkai0lHi3eof1-RwAlCkZSZWbHXUVCqUP3WeMR83sG3Mjles/s1600/5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="702" data-original-width="1149" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisloVqGopJpTGeEqTqJs-OW8uZzu-_8DXa5RHPwSTmKcJJVs12GsWyu693YVAMeZTqhhRKKYRy7oompxGxl9rpRKRNrL6Qqkai0lHi3eof1-RwAlCkZSZWbHXUVCqUP3WeMR83sG3Mjles/s640/5.JPG" width="640" /></a></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">dim(Output)[1] -> gives us no.of rows</span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">dim(Output)[2] -> gives us no.of columns</span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">In this example, if there are no columns I am assuming the sheet is empty and not doing any further processing.</span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br /></span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">More information on this topic, to be continued in <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_41.html" target="_blank"><i>part 5</i></a> ...</span></div>
<div style="background-color: white;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br />
</span> </div>
<h3 style="background-color: white; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">See Also:</span></h3>
</div>
<div class="MsoNormal" data-iceapc="4" data-iceapw="8" style="background-color: white; line-height: normal; margin-bottom: 0cm;">
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">SQL Server Export/Import excel using R script - Part 1</a></i></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">SQL Server Export/Import excel using R script - Part 2</a></i></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">SQL Server Export/Import excel using R script - Part 3</a></i></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://social.technet.microsoft.com/wiki/contents/articles/52528.sql-server-importexport-to-excel-using-r-script.aspx" target="_blank">My post on Technet Wiki - SQL Server Import/Export to Excel using R script</a></i></span></li>
</ul>
</div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-5251488690417463772018-12-25T16:20:00.000+05:302018-12-25T16:20:23.507+05:30SQL Server 2019 Installation<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how to install SQL Server 2019 CTP 2.2 Evaluation Edition on Windows 10 (64 bit) machine.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Download SQL Server 2019 for windows from link provided in the bottom of this post:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjtiinixIq55GuXaqrBN29srr_oDWPuvER8c-bqtzciZgxxVfRK8L4n244_P2rocyNiFethvZZg0p5lqBnHo75Opn8BEndj33JeIcI9F7R1fDDfCcNbtSo5OjKf94qoV1aSU2C_hhkM9GS/s1600/0.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="541" data-original-width="1237" height="139" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjtiinixIq55GuXaqrBN29srr_oDWPuvER8c-bqtzciZgxxVfRK8L4n244_P2rocyNiFethvZZg0p5lqBnHo75Opn8BEndj33JeIcI9F7R1fDDfCcNbtSo5OjKf94qoV1aSU2C_hhkM9GS/s320/0.JPG" width="320" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">After downloading, right click on the Setup file to Run as Administrator</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigDnD9sYIST5tBgcwRssTgHpSgygBDHBzK0CWpSYEHL2xwr0vru-j8gee1FSq7MBz2WK4ZQL-Cs9O2B5Ii05ly7hkjl_LoAg8Lm9FUMczYUVe10HseLudoIPxMOzIyw5gz_spmvYbiQN5w/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="674" data-original-width="845" height="255" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigDnD9sYIST5tBgcwRssTgHpSgygBDHBzK0CWpSYEHL2xwr0vru-j8gee1FSq7MBz2WK4ZQL-Cs9O2B5Ii05ly7hkjl_LoAg8Lm9FUMczYUVe10HseLudoIPxMOzIyw5gz_spmvYbiQN5w/s320/1.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Select the installation type, I did a Custom installation so that I can select the features to be installed:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ-ukiC2Qnp_tde-ntC4Vpje8I3Zq1MCNiZiv0ctD1zEpjftUppwNRbPAEOKgaYGLGWjwc6dWGZTv0v2IXkuGBF9Lac_D0yj7IErRZKf0AdPgm0g3JdAezUXlCOegOk4sGWAMD-lOWiIU-/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="671" data-original-width="845" height="254" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ-ukiC2Qnp_tde-ntC4Vpje8I3Zq1MCNiZiv0ctD1zEpjftUppwNRbPAEOKgaYGLGWjwc6dWGZTv0v2IXkuGBF9Lac_D0yj7IErRZKf0AdPgm0g3JdAezUXlCOegOk4sGWAMD-lOWiIU-/s320/2.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOOuHdkAFQ7q6c71RPqILpaMrY5Hii8Idn8tDDVJPgxMZL3E6wSFvdMQcc58wqbZn_aojAG-nA8EQN3FC84fqF1GF98ZK5y8lJ1ltFcqOinrOP3HgUhUgYLIxFp8r3PEjWZFZ4maT0ARgN/s1600/3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="669" data-original-width="843" height="253" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOOuHdkAFQ7q6c71RPqILpaMrY5Hii8Idn8tDDVJPgxMZL3E6wSFvdMQcc58wqbZn_aojAG-nA8EQN3FC84fqF1GF98ZK5y8lJ1ltFcqOinrOP3HgUhUgYLIxFp8r3PEjWZFZ4maT0ARgN/s320/3.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Click on Hardware and Software requirements. MSDN site will open, check whether the basic </span><span style="font-family: "arial" , "helvetica" , sans-serif;">requirements of your machine (like OS, disk space) is sufficient for installation.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh339tu2aE6sUp0zYta8OROHnWwk2kcsBca8w9TkBC6xOjWhV9arm2u9rSuVhsQWQkTttTFO0-nt3cm_PLYUVuDTMXQHkfPOMBjvyr4MW4YX_Seonkgoo8Q_KB1kVtvlD1sS2YH2N4rJEu4/s1600/4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="590" data-original-width="782" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh339tu2aE6sUp0zYta8OROHnWwk2kcsBca8w9TkBC6xOjWhV9arm2u9rSuVhsQWQkTttTFO0-nt3cm_PLYUVuDTMXQHkfPOMBjvyr4MW4YX_Seonkgoo8Q_KB1kVtvlD1sS2YH2N4rJEu4/s320/4.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Click on Installation - > New SQL Server stand-alone installation:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1UEMDwjytvncIQOy2zuH0oz46j23IQmEiilvSUOeES-iuO5TMrd0enhe1hNBAIDMBcin_U92Q3QLiGYCWSSEmacab1bGpdI3foozKnoBXPNIffGpCSFlffamG2Yw0mzxT877LO2o33HJF/s1600/5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="594" data-original-width="784" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1UEMDwjytvncIQOy2zuH0oz46j23IQmEiilvSUOeES-iuO5TMrd0enhe1hNBAIDMBcin_U92Q3QLiGYCWSSEmacab1bGpdI3foozKnoBXPNIffGpCSFlffamG2Yw0mzxT877LO2o33HJF/s320/5.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfeoEu2Xsnvg1Q_Y7a8eZC6BjYH1wA5cztgjEZFAcMgPX5CsMYuDvi-PxCh7FrjmLR70XG5zNZaMoqzhhvPwWIBAPc3zRRNcNfZC171-vqibc499F4-TCDzuqVcXrk6Cuomi0mEXcO6KUa/s1600/6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="603" data-original-width="804" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfeoEu2Xsnvg1Q_Y7a8eZC6BjYH1wA5cztgjEZFAcMgPX5CsMYuDvi-PxCh7FrjmLR70XG5zNZaMoqzhhvPwWIBAPc3zRRNcNfZC171-vqibc499F4-TCDzuqVcXrk6Cuomi0mEXcO6KUa/s320/6.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjksXDgkQ9bGEpV-B7bYydf2DpRtyly12uGWXeTQP2Ews7X9Ghb7afEFEceM7n8T-TIM6cpNzPSbDWcfkjmaa3NjQ8k7Ox0aqN7OCSVBJKpwBbE5xq9zb1IAf50BamYPOX2Vjr2B9Vyo-6C/s1600/7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="604" data-original-width="801" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjksXDgkQ9bGEpV-B7bYydf2DpRtyly12uGWXeTQP2Ews7X9Ghb7afEFEceM7n8T-TIM6cpNzPSbDWcfkjmaa3NjQ8k7Ox0aqN7OCSVBJKpwBbE5xq9zb1IAf50BamYPOX2Vjr2B9Vyo-6C/s320/7.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can ignore the warnings: </span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSWkYDLdmlDGCSpjT5P8eYaH-KhvaU6uwxwd_ZIHJyBSk3wWuXg2wr2oEhZ9Rsqhd7OvyCLi31PNNzge7JdOWd1YCSScvG8tp4kREzV5s8M7BIGVSwUk9N3RkhR7lzv-yj1012d94xtGoe/s1600/8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="609" data-original-width="805" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSWkYDLdmlDGCSpjT5P8eYaH-KhvaU6uwxwd_ZIHJyBSk3wWuXg2wr2oEhZ9Rsqhd7OvyCLi31PNNzge7JdOWd1YCSScvG8tp4kREzV5s8M7BIGVSwUk9N3RkhR7lzv-yj1012d94xtGoe/s320/8.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Except Polybase Query Service, I have selected all the features.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">As Polybase Query Service requires Oracle JRE which is not installed on my machine.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsoDWYd71ApwREvmM4ZNBSLS5NZ8zFX6eQyRjS8R0C0z6DSk8LvbvocuV7RICkuN13ElxO21FOXnussT9IlGMz7A-YTFmOAwGXlktNNa6ZrDkirmBbGR2JcRwtOs72UnXsjdt_pZhWSa7E/s1600/9.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="606" data-original-width="806" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhsoDWYd71ApwREvmM4ZNBSLS5NZ8zFX6eQyRjS8R0C0z6DSk8LvbvocuV7RICkuN13ElxO21FOXnussT9IlGMz7A-YTFmOAwGXlktNNa6ZrDkirmBbGR2JcRwtOs72UnXsjdt_pZhWSa7E/s320/9.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Provide the Instance name if there are any other version of SQL Server already installed on same machine or give the default instance.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTeVFJlznDadnnERoultKBJ1_F5Kvbpn4YPE7p0lCL6HNgMxDloQ2sFp1yRFV99WzDq9EgBlDyDE7jsz_haERgSb2Jnjan9LE-sSRx2tbVsDvkiqvEZv6iBmDfrFXqn_PNkSU7fljo7AWo/s1600/10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="609" data-original-width="808" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTeVFJlznDadnnERoultKBJ1_F5Kvbpn4YPE7p0lCL6HNgMxDloQ2sFp1yRFV99WzDq9EgBlDyDE7jsz_haERgSb2Jnjan9LE-sSRx2tbVsDvkiqvEZv6iBmDfrFXqn_PNkSU7fljo7AWo/s320/10.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh05GZQM1SsZ9iXbi2ly_laGQ22LX66wPY5_T-lySJzRhS7YnjcJzsXcHKXEbS0QWSXc3df9WNR0jSdxvChldXit3YUgVLtJkCE6og_f64XFHoGoqrOMKrEtUA36j-QHeWvLdYzvWexNuqy/s1600/11.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="609" data-original-width="803" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh05GZQM1SsZ9iXbi2ly_laGQ22LX66wPY5_T-lySJzRhS7YnjcJzsXcHKXEbS0QWSXc3df9WNR0jSdxvChldXit3YUgVLtJkCE6og_f64XFHoGoqrOMKrEtUA36j-QHeWvLdYzvWexNuqy/s320/11.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Select the authentication type and Add Current User as SQL Server administrators:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZPfUfU9rL-J2BoO_jnBIS9LoqSdOS4lsrCGc65s6SW87p7ENGVRFc4Sg9LsYaixFZWZ2HdNGkJtrtmpCEOAj3idMu92MM0tAD0VGtghg8v4k7LVq1kPUILBBuVkdjfLN0d3k_4kJLm2_2/s1600/12.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="608" data-original-width="803" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZPfUfU9rL-J2BoO_jnBIS9LoqSdOS4lsrCGc65s6SW87p7ENGVRFc4Sg9LsYaixFZWZ2HdNGkJtrtmpCEOAj3idMu92MM0tAD0VGtghg8v4k7LVq1kPUILBBuVkdjfLN0d3k_4kJLm2_2/s320/12.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi66qEv4lyB4esOhFOD27ms-EHsh2qSXpMONcRqmH4wdKQBK9fdXwQCls2jJ0RzizsK63_IgtGR6Fjjl-2b88bXMjlzmnEx881Gr3I38PW0U6VK9a0Pfk9c8N_VLlIdye1FQKqdwuCJSFQj/s1600/13.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="608" data-original-width="802" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi66qEv4lyB4esOhFOD27ms-EHsh2qSXpMONcRqmH4wdKQBK9fdXwQCls2jJ0RzizsK63_IgtGR6Fjjl-2b88bXMjlzmnEx881Gr3I38PW0U6VK9a0Pfk9c8N_VLlIdye1FQKqdwuCJSFQj/s320/13.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">SSIS Scale out configuration:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmkL4aK3Rg_Kv71IS_ip9XzKJuAX0vuebX50nqtQN74g64JIJPT7B1DvsUcjyqA16x2Kun0UObRoja8Ebux0ejFB9Ah3IPXNNv9wKpbKCwAJ546pZXBzkSyIQCCtl4Ma2tuQhWOFiQxUAJ/s1600/14.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="605" data-original-width="802" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmkL4aK3Rg_Kv71IS_ip9XzKJuAX0vuebX50nqtQN74g64JIJPT7B1DvsUcjyqA16x2Kun0UObRoja8Ebux0ejFB9Ah3IPXNNv9wKpbKCwAJ546pZXBzkSyIQCCtl4Ma2tuQhWOFiQxUAJ/s320/14.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9ZChoVcrXvHwJ_IkDhZXRMNNPIVJEheq3jJZDYdZmE7Wu7ji4_Pgo9EOIr5t8I6x4dz0Ls8DhhHEbxNKAFCw-MztNXdKT5tiJ2a2D9XQuSkM0TpXZDunLhJu3yoMcnvE0c9rNuzKpVL9V/s1600/15.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="607" data-original-width="801" height="242" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9ZChoVcrXvHwJ_IkDhZXRMNNPIVJEheq3jJZDYdZmE7Wu7ji4_Pgo9EOIr5t8I6x4dz0Ls8DhhHEbxNKAFCw-MztNXdKT5tiJ2a2D9XQuSkM0TpXZDunLhJu3yoMcnvE0c9rNuzKpVL9V/s320/15.JPG" width="320" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Accept the consent to install R & Python:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDCQ8fC4xWh-VW-87oLkoNZq_S3zaLUdyzrfZuHHVyQGALnxMrtkDoz6I4TrWYOokkljHvdS4npPAl2gdqYcS8Few2nMfahUBXDbl2mslVEjQu5q-5whaa1BT9NYM-Q9FiaBVG_SxBCUuO/s1600/16.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="605" data-original-width="803" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDCQ8fC4xWh-VW-87oLkoNZq_S3zaLUdyzrfZuHHVyQGALnxMrtkDoz6I4TrWYOokkljHvdS4npPAl2gdqYcS8Few2nMfahUBXDbl2mslVEjQu5q-5whaa1BT9NYM-Q9FiaBVG_SxBCUuO/s320/16.JPG" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyVEouSDQry7hVdiiYV_I7DtD9f1Sezu4S8xFmySmxa3LRnp5xOawIWUPpur84cqHpaqXTVG4roNezO0Opt4Yac-qwK7G4q-XQZni6UQRCrRx7oyg58CyI_q_VzCsfEiQ8xWPm3Pc_Moqn/s1600/17.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="608" data-original-width="807" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyVEouSDQry7hVdiiYV_I7DtD9f1Sezu4S8xFmySmxa3LRnp5xOawIWUPpur84cqHpaqXTVG4roNezO0Opt4Yac-qwK7G4q-XQZni6UQRCrRx7oyg58CyI_q_VzCsfEiQ8xWPm3Pc_Moqn/s320/17.JPG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">After clicking on Install, it took around 30 mins.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheJgQaNG5i3hM8vGIVS_hT4RF8yl8d62MZc8Y8r7qgh9VSOIlezk5W9z8lwzFWCp3E2UP2jQ1fBoLzExrK-movoj70gasHbLbPvOVMxZ8nIcIfzhNj5tOrePUqa1KqYYJtGmA-huPv9n9l/s1600/18.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="609" data-original-width="806" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEheJgQaNG5i3hM8vGIVS_hT4RF8yl8d62MZc8Y8r7qgh9VSOIlezk5W9z8lwzFWCp3E2UP2jQ1fBoLzExrK-movoj70gasHbLbPvOVMxZ8nIcIfzhNj5tOrePUqa1KqYYJtGmA-huPv9n9l/s320/18.JPG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Some of the tools & services are not installed as part of above installation.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Open the Set up file -> Installation (select the tool or service to be installed)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">(or) we can download it from web & install it separately.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I tried to install SSMS tool (by clicking on options shown in below screenshot):</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">MSDN site opened in IE with the download link.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjact-D5jCFg1ntF-dr1aQHezLWfpo0BHjCTSpVupSS9ALn5yV9jUJtSW-c7duTcClM2JoFFl0oWoU0sHnYOxBzfqE6KAPhhHMS4nCARQoao9MwCd3wzmLtui7CUGpo3MrQy6sADKco71qg/s1600/19.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="592" data-original-width="785" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjact-D5jCFg1ntF-dr1aQHezLWfpo0BHjCTSpVupSS9ALn5yV9jUJtSW-c7duTcClM2JoFFl0oWoU0sHnYOxBzfqE6KAPhhHMS4nCARQoao9MwCd3wzmLtui7CUGpo3MrQy6sADKco71qg/s320/19.JPG" width="320" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">What's new in <a href="https://docs.microsoft.com/en-us/sql/sql-server/what-s-new-in-sql-server-ver15?view=sqlallproducts-allversions" target="_blank"><i>SQL Server 2019</i></a></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">SQL Server 2019 can be downloaded from <i><a href="https://www.microsoft.com/en-us/sql-server/sql-server-2019" target="_blank">here</a>. </i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><br /></i></span>
<br />
<div style="text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: x-small;"><i>Merry christmas!!</i></span></div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-83361198156370037252018-12-24T00:07:00.001+05:302019-07-06T01:37:35.178+05:30SQL Server Export/Import excel using R script - Part 3<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white;">In continuation to my previous posts on </span><i>SQL Server Export/Import excel using R script - <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">Part 1</a> & <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">Part 2</a>,</i> where we saw about exporting to Excel. In this post, let us see how to import Excel into SQL Server table using R script. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">In <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank"><i>part 2</i></a> post, I have exported few tables under "Sales" schema from "WideWorldImporters" sample database into single excel file with six sheets: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Sheet 1 - CustomerTransactions</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sheet 2 - InvoiceLines</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sheet 3 - Invoices</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sheet 4 - OrderLines</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sheet 5 - Orders</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sheet 6 - SpecialDeals</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">While importing we have two scenarios 1) Excel file structure is defined & known</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> 2) Excel file structure is unknown and table has to be created dynamically</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Let us see example for both scenarios:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <b><span style="font-family: "arial" , "helvetica" , sans-serif;">Scenario 1: </span><span style="font-family: "arial" , "helvetica" , sans-serif;">Excel file structure is defined & known</span></b><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">For this scenario, let us try to import sheet 1 by setting the argument sheetName = 1 in read.xlsx2 function. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">As we know the file structure, table has been </span><span style="font-family: "arial" , "helvetica" , sans-serif;">created beforehand.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">Please note: In the below script, we need to configure the values, </span><span style="font-family: arial, helvetica, sans-serif;">pass folder path details for read.xlsx2 function</span><br />
<span style="font-family: arial, helvetica, sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<br />
<br />
<span style="background-color: #3d85c6;"><span style="color: white;">SET NOCOUNT ON</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">DROP TABLE IF EXISTS CustomerTransactions_Import</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br />
</span></span> <span style="background-color: #3d85c6;"><span style="color: white;">--As we know the Excel file structure beforehand, creating an empty table to import the data from Excel sheet</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">SELECT NULL AS RowId,* </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">INTO CustomerTransactions_Import </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">FROM [Sales].[CustomerTransactions]</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">WHERE 1 = 2</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br />
</span></span> <span style="background-color: #3d85c6;"><span style="color: white;">DECLARE @RScript NVARCHAR(MAX) =N' </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> require(rJava);</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>require("xlsx");</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> OutputDataSet <- read.xlsx2("H:/Temp/Blog/ImportExportExcel/Sales.xlsx", sheetName = 1,header=TRUE)</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><span style="white-space: pre;"> </span>'</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">--Inserting from Excel</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">INSERT dbo.CustomerTransactions_Import</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">EXEC sp_execute_external_script</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> @language = N'R'</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"> ,@script = @RScript</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br />
</span></span> <span style="background-color: #3d85c6;"><span style="color: white;">--SELECT * FROM CustomerTransactions_Import</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br />
</span></span> <span style="background-color: #3d85c6;"><span style="color: white;">--Below query compares the data in source table and data imported from Excel</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">SELECT Src.CustomerTransactionID,Imp.CustomerTransactionID </span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">FROM [Sales].[CustomerTransactions] Src</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">LEFT JOIN CustomerTransactions_Import Imp</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;">ON Src.CustomerTransactionID = Imp.CustomerTransactionID</span></span><br />
<span style="background-color: #3d85c6;"><span style="color: white;"><br /></span></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4-C_22QU3gSrd7aaR21LrIQuTuyPxcX0H3QTY63BScZOa7To-zQPO9tACMmEYJv4zDEVymvwepBGDdmNO62blq0-HBtdKBzqCQbdG5iFUXQA4LUsK3HCRujsA9sd4S20FCYJ8ykWaBE2M/s1600/R10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="630" data-original-width="1149" height="350" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4-C_22QU3gSrd7aaR21LrIQuTuyPxcX0H3QTY63BScZOa7To-zQPO9tACMmEYJv4zDEVymvwepBGDdmNO62blq0-HBtdKBzqCQbdG5iFUXQA4LUsK3HCRujsA9sd4S20FCYJ8ykWaBE2M/s640/R10.JPG" width="640" /></a></div>
<span style="background-color: #3d85c6;"><span style="color: white;"><br /></span></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Scenario 2: Excel file structure is unknown and table has to be created dynamically</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For this scenario, let us try to import sheet 2 by setting the argument sheetName = 2 in read.xlsx2 function. </span><span style="font-family: "arial" , "helvetica" , sans-serif;">As we don't know the file structure, table will be built dynamically using </span><span style="font-family: "arial" , "helvetica" , sans-serif;">RevoScaleR function - rxDataStep and imported with data from read.xlsx2 function.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">Please note: In the below script, we need to configure the values for </span><br />
<span style="font-family: arial, helvetica, sans-serif;">sqlConnString variable with database connection details</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: arial, helvetica, sans-serif;">Pass folder path details for read.xlsx2 function</span><br />
<div>
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;">DECLARE @RScript NVARCHAR(MAX) =N' </span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre;"> </span>Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> require(rJava);</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre;"> </span>require("xlsx");</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> Output <- read.xlsx2("H:/Temp/Blog/ImportExportExcel/Sales.xlsx", sheetName = 2,header=TRUE)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre;"> </span>sqlConnString <- "Driver=SQL Server;Server=''LAPTOP-LCUB6HBB''; Database=WideWorldImporters;Uid=sa;Pwd=***"</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlTable <- "InvoiceLines_Import"</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><span style="white-space: pre;"> </span>if (rxSqlServerTableExists(sqlTable, connectionString = sqlConnString)) rxSqlServerDropTable(sqlTable, connectionString = sqlConnString)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> sqlDS <- RxSqlServerData(connectionString = sqlConnString,table = sqlTable)</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> rxDataStep(inData = Output, outFile = sqlDS,overwrite = TRUE)'</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;">EXEC sp_execute_external_script</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> @language = N'R'</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"> ,@script = @RScript</span><br />
<span style="background-color: #3d85c6; color: white; font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglOpKF16Q8bmZqbdiCSCGtKpOknbsgu_HwDd8fWFyXeblEbAHhak0r3HtWml548jo1QdKoQed2cyyrtN67bQGTaGUNj1T9CManopMAqop8nCOs7NDryIYBU87rVuM7oHHEDNS1g1gdQMXT/s1600/R11.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="328" data-original-width="1071" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglOpKF16Q8bmZqbdiCSCGtKpOknbsgu_HwDd8fWFyXeblEbAHhak0r3HtWml548jo1QdKoQed2cyyrtN67bQGTaGUNj1T9CManopMAqop8nCOs7NDryIYBU87rVuM7oHHEDNS1g1gdQMXT/s640/R11.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">sqlConnString variable needs to be set with ServerName, DatabaseName, Username & Password for ODBC SQL Server connection.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">sqlTable variable is set with the name of table to be created.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"> More information on this topic, to be continued in <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_25.html" target="_blank"><i>part 4</i></a> ...</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>See Also:</i></span></h3>
<ul style="text-align: left;">
<li><i style="font-family: arial, helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">SQL Server Export/Import excel using R script - Part 1</a></i></li>
<li><i style="font-family: arial, helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">SQL Server Export/Import excel using R script - Part 2</a></i></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://social.technet.microsoft.com/wiki/contents/articles/52528.sql-server-importexport-to-excel-using-r-script.aspx" target="_blank">My post on Technet Wiki - SQL Server Import/Export to Excel using R script</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-86609022236184287322018-12-23T03:37:00.005+05:302019-02-10T17:40:50.058+05:30SQL Server Export/Import excel using R script - Part 2<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: black;">In continuation to my previous post on <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank"><i>SQL Server Export/Import excel using R script - Part 1</i></a>, let us see few other examples in this post.</span><span style="color: black;"><o:p></o:p></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: black;"><br />
In previous post, I had selected <u>only first six key fields</u> from [WideWorldImporters].[Sales].[Orders] table and passed this as inputdataset (dataframe) to write.xlsx2 function.</span><span style="color: black;"><o:p></o:p></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="color: black;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Instead if I do "</span><span style="font-family: "courier new" , "courier" , monospace;">SELECT * FROM </span></span></span><span style="font-family: "courier new" , "courier" , monospace;">[WideWorldImporters].[Sales].[Orders]"</span><span style="font-family: "arial" , "helvetica" , sans-serif;">, I got error as shown in below screenshot:</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="color: black;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOtJS0Cj6wti1Szq6QAeapRCDA5siNOqcsIp8zvFN-o6aMc6xgYDNanOnPQ25bW-Igv3zA1x5gQZex7txr1fWpTn53sSMZwMMgEIn5phtm5lZYw2yAxemY-lgR7L0PpkZj6on1owEI0Jp3/s1600/R8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="626" data-original-width="1248" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOtJS0Cj6wti1Szq6QAeapRCDA5siNOqcsIp8zvFN-o6aMc6xgYDNanOnPQ25bW-Igv3zA1x5gQZex7txr1fWpTn53sSMZwMMgEIn5phtm5lZYw2yAxemY-lgR7L0PpkZj6on1owEI0Jp3/s640/R8.JPG" width="640" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"></span></div>
<a name='more'></a><br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">This is because data type conversions is getting performed implicitly and some of the data types </span><span style="font-family: "arial" , "helvetica" , sans-serif;">are not supported when data is passed between R libraries and SQL Server.</span></div>
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">To workaround this, we can ignore the fields from exporting that has data types that are <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">not supported and also doesn't make any sense exporting (for example field with image data type)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">To workaround the issue of data types that are poorly converted when data is passed between </span><span style="font-family: "arial" , "helvetica" , sans-serif;">R and SQL Server, we can make use of TRY_CONVERT(VARCHAR(MAX),field name</span><span style="font-family: "arial" , "helvetica" , sans-serif;">)</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">So in below example, I am going to export all the tables data (non temporal tables) that falls under </span><span style="font-family: "arial" , "helvetica" , sans-serif;">"Sales" schema from WideWorldImporters sample database.</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"></span></div>
<pre class="brush:sql">USE [WideWorldImporters]
GO
SET NOCOUNT ON
DROP TABLE IF EXISTS #ExportTablesList, #FinalExportList
SELECT CONCAT('TRY_CONVERT(','VARCHAR(MAX),',C.name,') AS ',QUOTENAME(C.NAME)) Columns -- To cover poor data type conversions
,T.name TableName
,DENSE_RANK()OVER(ORDER BY T.name) TableCount
,TY.name DataType
INTO #ExportTablesList
FROM Sys.tables T
JOIN sys.columns C
ON T.object_id = C.object_id
JOIN sys.types TY
ON C.[user_type_id] = TY.[user_type_id]
WHERE Schema_name(T.schema_id) = 'Sales'
AND temporal_type = 0 --NON_TEMPORAL_TABLE
AND TY.name <> 'image' -- Ignore the datatypes that are not required to be exported
SELECT TableName,MAX(TableCount) TableCount
, STUFF(
(
SELECT ', ' + C.Columns
From #ExportTablesList As C
WHERE C.TableName = T.TableName
FOR XML PATH('')
), 1, 2, '') AS Columns
INTO #FinalExportList
From #ExportTablesList As T
GROUP BY TableName
DECLARE @I INT = 1
,@TableName NVARCHAR(200)
,@SQL NVARCHAR(MAX) = N''
,@RScript NVARCHAR(MAX) = N''
WHILE @I <= (SELECT MAX(TableCount) FROM #FinalExportList)
BEGIN
SELECT @SQL = CONCAT('SELECT TOP 10 ',Columns,' FROM [WideWorldImporters].[Sales].',QUOTENAME(TableName),';')
,@TableName = TableName
FROM #FinalExportList WHERE TableCount = @I
SET @RScript = CONCAT('
Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")
library(rJava);
library("xlsx");
write.xlsx2(InputDataSet, file = "H:/Temp/Blog/ImportExportExcel/Sales.xlsx", sheetName= "',@TableName,'",
col.names=TRUE, append=TRUE) ')
EXEC sp_execute_external_script
@language = N'R'
,@script = @RScript
,@input_data_1 = @SQL
SET @I = @I + 1
END
</pre>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In above example, I have tried to export into single excel file under multiple sheets with sheet name </span><span style="font-family: "arial" , "helvetica" , sans-serif;">as table name.</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><span style="font-family: "arial" , "helvetica" , sans-serif;">This is achieved by setting argument "append=TRUE" in write.xlsx2 function.</span><br />
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
</div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">While exporting multiple tables data, if we need any specific configurations to be done say like </span><span style="font-family: "arial" , "helvetica" , sans-serif;">export top 100 records, order by key field or filter on particular field - all these can be done </span><span style="font-family: "arial" , "helvetica" , sans-serif;">by tweaking the SQL that is built dynamically (like I have done in above example).</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"> </span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyDbtNCk2N9ejcvW4lt-SH-Hs2l_BVeTzn90pgaYY2VCI6RGVMSiZhGug4ChiUVAP7i5p1BkIAumBvrDbdMHwjJ1sqtdY3c1zTfwfa2cNuGYdkf4Y0RC__ui_P4AKH9PVVOi_wyuoYUv72/s1600/R9.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="687" data-original-width="1123" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyDbtNCk2N9ejcvW4lt-SH-Hs2l_BVeTzn90pgaYY2VCI6RGVMSiZhGug4ChiUVAP7i5p1BkIAumBvrDbdMHwjJ1sqtdY3c1zTfwfa2cNuGYdkf4Y0RC__ui_P4AKH9PVVOi_wyuoYUv72/s640/R9.JPG" width="640" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Similarly it can also be exported to different excel files instead of different tabs within single sheet with dynamic file names by tweaking </span><span style="font-family: "arial" , "helvetica" , sans-serif;">the append & file arguments in write.xlsx2 function. Below script does this:</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<pre class="brush:sql">USE [WideWorldImporters]
GO
SET NOCOUNT ON
DROP TABLE IF EXISTS #ExportTablesList, #FinalExportList
SELECT CONCAT('TRY_CONVERT(','VARCHAR(MAX),',C.name,') AS ',QUOTENAME(C.NAME)) Columns -- To cover poor data type conversions
,T.name TableName
,DENSE_RANK()OVER(ORDER BY T.name) TableCount
,TY.name DataType
INTO #ExportTablesList
FROM Sys.tables T
JOIN sys.columns C
ON T.object_id = C.object_id
JOIN sys.types TY
ON C.[user_type_id] = TY.[user_type_id]
WHERE Schema_name(T.schema_id) = 'Sales'
AND temporal_type = 0 --NON_TEMPORAL_TABLE
AND TY.name <> 'image' -- Ignore the datatypes that are not required to be exported
SELECT TableName,MAX(TableCount) TableCount
, STUFF(
(
SELECT ', ' + C.Columns
From #ExportTablesList As C
WHERE C.TableName = T.TableName
FOR XML PATH('')
), 1, 2, '') AS Columns
INTO #FinalExportList
From #ExportTablesList As T
GROUP BY TableName
DECLARE @I INT = 1
,@TableName NVARCHAR(200)
,@SQL NVARCHAR(MAX) = N''
,@RScript NVARCHAR(MAX) = N''
WHILE @I <= (SELECT MAX(TableCount) FROM #FinalExportList)
BEGIN
SELECT @SQL = CONCAT('SELECT TOP 10 ',Columns,' FROM [WideWorldImporters].[Sales].',QUOTENAME(TableName),';')
,@TableName = TableName
FROM #FinalExportList WHERE TableCount = @I
SET @RScript = CONCAT('
Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")
library(rJava);
library("xlsx");
write.xlsx2(InputDataSet, file = "H:/Temp/Blog/ImportExportExcel/',@TableName,'.xlsx", sheetName= "',@TableName,'",
col.names=TRUE, append=FALSE) ')
EXEC sp_execute_external_script
@language = N'R'
,@script = @RScript
,@input_data_1 = @SQL
SET @I = @I + 1
END
</pre>
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span> <span style="font-family: "arial" , "helvetica" , sans-serif;"> </span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_bq0ainimANlh3VsonareRBeLgDYS7bQIaz9EnJiXCl39_n8CxEP62ZVGMQaE43yKsc_8OTU8byuUVP_-0gpshRa2XZeSNFZqaNzLWfOQtgwAxSLStZzCkyN8AAmnrskaIzmfWUELmuWf/s1600/Part2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="249" data-original-width="662" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_bq0ainimANlh3VsonareRBeLgDYS7bQIaz9EnJiXCl39_n8CxEP62ZVGMQaE43yKsc_8OTU8byuUVP_-0gpshRa2XZeSNFZqaNzLWfOQtgwAxSLStZzCkyN8AAmnrskaIzmfWUELmuWf/s640/Part2.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"> </span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">More information on this topic, to be continued in <i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_24.html" target="_blank">part 3</a></i> ...</span><span style="font-family: "courier new"; font-size: 10pt;"><o:p></o:p></span> <span style="font-family: "arial" , "helvetica" , sans-serif;"> </span> <span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-outline-level: 3;">
</div>
<h3 style="line-height: normal; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b><i>See Also: </i></b></span><b><i><span style="font-family: "arial" , "helvetica" , sans-serif;"> </span></i></b></h3>
<div>
<ul style="text-align: left;">
<li><i style="font-family: arial, helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r.html" target="_blank">SQL Server Export/Import excel using R script - Part 1</a></i></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><i><a href="https://social.technet.microsoft.com/wiki/contents/articles/52528.sql-server-importexport-to-excel-using-r-script.aspx" target="_blank">My post on Technet Wiki - SQL Server Import/Export to Excel using R script</a></i></span></li>
</ul>
</div>
<h3 style="line-height: normal; text-align: left;">
<b><i><span style="font-family: "arial" , "helvetica" , sans-serif;">Reference: </span></i></b></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: 0.0001pt; text-align: left;">
</div>
<ul style="text-align: left;" type="disc">
<li class="MsoNormal" style="line-height: normal;"><span style="font-family: "times new roman" , serif;"><i><span style="color: blue; font-family: "arial" , sans-serif;"><a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/r/r-libraries-and-data-types?view=sql-server-2017" target="_blank">Data type mappings between R and SQL Server</a></span></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-58875599533257280752018-12-23T03:37:00.001+05:302019-07-06T01:26:52.446+05:30SQL Server Export/Import excel using R script - Part 1<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how we can easily import or export to excel with just few lines of r script executed from SSMS using sp_execute_external_script.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have done below example using SQL Server 2016 on windows 10 (64 bit) and R (3.2.2).<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Refer my previous post on how to <i><span style="color: blue;"><a href="http://www.allaboutmssql.com/2017/02/sql-server-2016-getting-started-with-r.html" target="_blank">install R services</a></span></i>. <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
Also I have used "<i><span style="color: blue;"><a href="https://cloudblogs.microsoft.com/sqlserver/2016/06/09/wideworldimporters-the-new-sql-server-sample-database/" target="_blank">WideWorldImporters</a></span></i>" sample database for this example.</span><span style="font-family: "times new roman" , serif; font-size: 12.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , sans-serif; font-size: 12.0pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , sans-serif; font-size: 12.0pt;"></span></div>
<a name='more'></a><br />
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Now let us jump on to the prerequisite steps for this approach. Though prerequisite steps are bit heavy, it is one time and thereafter enable us to export / import to excel easily with just few lines of code.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
Prerequisite Step 1: Check the version of R by running <span style="background: #3d85c6; color: white;">sessionInfo()</span> from R.exe<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
In my case, I have installed SQL Server 2016 so R.exe will be found in below path:<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="background: black; color: white; line-height: 107%;"><span style="font-family: "arial" , "helvetica" , sans-serif;">C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\R_SERVICES\bin</span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="background: black; color: white; font-family: "arial" , sans-serif; font-size: 12.0pt; line-height: 107%;"><br />
</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj69vWRY-F_U8SFn7HaHjEbmx0R4kRziuLL3kAHAdyKbmtfJWvU5WnyXOYBfdFzHpw3Z7pV33asrb4jn5QRO9VUPlGRVseVqQ7IydF56C6otJdWzd5cXLVtjU_xfRPsr82g8pKPJO2h_whQ/s1600/R1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="239" data-original-width="490" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj69vWRY-F_U8SFn7HaHjEbmx0R4kRziuLL3kAHAdyKbmtfJWvU5WnyXOYBfdFzHpw3Z7pV33asrb4jn5QRO9VUPlGRVseVqQ7IydF56C6otJdWzd5cXLVtjU_xfRPsr82g8pKPJO2h_whQ/s640/R1.JPG" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Prerequisite Step 2: From above step I got to know that it is 64 bit, So I have downloaded Java JDK for 64 bit (.exe file) from <i><span style="color: blue;"><a href="https://www.oracle.com/technetwork/java/javase/downloads/jdk10-downloads-4416644.html" target="_blank">here</a></span></i> & installed it.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background: black; color: white;">C:\Program Files\Java\jdk-11.0</span><span style="color: white;">.</span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Prerequisite Step 3: Now install rJava package from R.exe<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: rgb(61 , 133 , 198); color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">install.packages("rJava")</span></span><span style="font-family: "courier new"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: #3d85c6; color: white; font-family: "arial" , sans-serif; font-size: 10.0pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: #3d85c6; color: white; font-family: "arial" , sans-serif; font-size: 10.0pt;"><br />
</span></div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNqdRdAuDlE46f7VGLaFI5sXjj832lZbLLiSDzYV8lN5qMwnrvJyCRos3vEQTulbr0z97MCN6ujeZSWj66iKcKZTfsA8pOFxta4DzpGJJ5p6kcdY36PkMCdgkk7wdVjTmky05amG4boRBH/s1600/R2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="587" data-original-width="876" height="428" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNqdRdAuDlE46f7VGLaFI5sXjj832lZbLLiSDzYV8lN5qMwnrvJyCRos3vEQTulbr0z97MCN6ujeZSWj66iKcKZTfsA8pOFxta4DzpGJJ5p6kcdY36PkMCdgkk7wdVjTmky05amG4boRBH/s640/R2.JPG" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Prerequisite Step 4: Now let us set the JAVA_HOME environment to Java JDK installed in Step 2 and </span><span style="font-family: "arial" , "helvetica" , sans-serif;">load the rJava package to check if it working from R.exe.</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: rgb(61 , 133 , 198); color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")<o:p></o:p></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: rgb(61 , 133 , 198); color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">library(rJava)</span></span><span style="font-family: "courier new"; font-size: 10.0pt;"><o:p></o:p></span></div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie5_OjmEueOx7RpHE4rNj7Fgx4iqs3X_w6OWDP20HnIeQKDJK6wLjzEDZGzLbLAdttyur2JW6qhyphenhyphenxDIiyOdSTwsTLnRcbVB2fQAZ_MiHBZv9xYKniR5IprCvRy-QiR3Lgh_tFWE7jCQeKE/s1600/R3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="462" data-original-width="501" height="590" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie5_OjmEueOx7RpHE4rNj7Fgx4iqs3X_w6OWDP20HnIeQKDJK6wLjzEDZGzLbLAdttyur2JW6qhyphenhyphenxDIiyOdSTwsTLnRcbVB2fQAZ_MiHBZv9xYKniR5IprCvRy-QiR3Lgh_tFWE7jCQeKE/s640/R3.JPG" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Prerequisite Step 5: Now it's time to install & load the xlsx package from R.exe.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-left: 36.0pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: "arial" , "helvetica" , sans-serif;">·<span style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><!--[endif]-->xlsx package is one of the powerful R packages to read, write and format Excel files.</span></div>
<div class="MsoNormal" style="line-height: normal; margin-left: 36.0pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: "arial" , "helvetica" , sans-serif;">·<span style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><!--[endif]-->It is a java-based solution and it is available for Windows, Mac and Linux.</span></div>
<div class="MsoNormal" style="line-height: normal; margin-left: 36.0pt; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: "arial" , "helvetica" , sans-serif;">·<span style="font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><!--[endif]-->It works for both Excel 2007 and Excel 97/2000/XP/2003 file formats (xls and xlsx file formats).<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 36.0pt; margin-right: 0cm; margin-top: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: rgb(61 , 133 , 198); color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">install.packages("xlsx")<o:p></o:p></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="background: rgb(61 , 133 , 198); color: white;"><span style="font-family: "arial" , "helvetica" , sans-serif;">library("xlsx")</span></span><span style="font-family: "courier new"; font-size: 10.0pt;"><o:p></o:p></span></div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhoPoyjYO0xw6ZUc-BDvZ6o0AetIEbroVwMdx_sHPDssKOiN0mkhkT0gJo6lXx2KTpXeTBno2_hA_EWV3AzJPWk7e8gLOOSCUKKlrYKPyC3b-gq5dUjHNLZy-70f56Rfq47r3FNH4qYQlw/s1600/R4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="727" data-original-width="900" height="516" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhoPoyjYO0xw6ZUc-BDvZ6o0AetIEbroVwMdx_sHPDssKOiN0mkhkT0gJo6lXx2KTpXeTBno2_hA_EWV3AzJPWk7e8gLOOSCUKKlrYKPyC3b-gq5dUjHNLZy-70f56Rfq47r3FNH4qYQlw/s640/R4.JPG" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Prerequisite Step 6: Copy & paste the packages downloaded from above steps into default R library to SQL Server R_services library.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have copied <b>xlsx,xlsxjars & rJava</b> folders from <span style="background: black; color: white;">C:\Users\Sathy\Documents\R\win-library\3.2</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">to <span style="background: black; color: white;">C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\R_SERVICES\library</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<span style="font-family: arial, helvetica, sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: arial, helvetica, sans-serif;">As a Pre-requisite, right-click on the folder where Excel files will be imported / exported and provide folder access (read, write) to "ALL_APPLICATION_PACKAGES".</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Now as part of last prerequisite step, from SSMS let us check the installed R packages. To execute R script using sp_execute_external script, refer to my previous post link provided in the beginning of this post.</span><span style="font-family: "times new roman" , serif; font-size: 12.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
</div>
<span style="background-color: #3d85c6;"><span style="color: white; font-family: "arial" , "helvetica" , sans-serif;">sp_execute_external_script @language = N'R', @script = N'OutputDataSet <- data.frame(installed.packages())'</span></span><br />
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
</div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "courier new"; font-size: 10.0pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "courier new"; font-size: 10.0pt;"><br />
</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHTLIeAvz7wecS1d1g8PhbeVR_872J27JEz-NPWzs4D7cdu2ZJX9mP8OoTiH-5ae7q6ADSW0VR2isvNuSa-P9_63zheBgHs53X3IYjIg4O9ncmokX1NsfDFubgnW5NRHnh9OESBUKVLUYh/s1600/R5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="594" data-original-width="979" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHTLIeAvz7wecS1d1g8PhbeVR_872J27JEz-NPWzs4D7cdu2ZJX9mP8OoTiH-5ae7q6ADSW0VR2isvNuSa-P9_63zheBgHs53X3IYjIg4O9ncmokX1NsfDFubgnW5NRHnh9OESBUKVLUYh/s640/R5.JPG" width="640" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "courier new"; font-size: 10.0pt;"><br />
</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "courier new"; font-size: 10.0pt;"><br />
</span></div>
</div>
<div>
</div>
<div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Now the script to export SQL Server table data into Excel: <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below script is pretty much simple and I have selected <u>only first six key fields</u> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">from [WideWorldImporters].[Sales].[Orders] table and passed this as inputdataset (dataframe) to write.xlsx2 function.</span><span style="font-family: "arial" , sans-serif; font-size: 13.5pt;"><o:p></o:p></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 13.5pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvLLslwOmDZHxv7ykDv21pniMWT4sK4pjZgDQetyiLvMk_WHvqq_BhAxLO8V9ZX8kVUwz47e6VwaR2A1Kr4lhnHkawqzDUNduXKnrZg2IzjNlEKkcdcc3fv1jMpzAFEc9Dd3OJBOoDjBIj/s1600/R6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="701" data-original-width="1362" height="328" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvLLslwOmDZHxv7ykDv21pniMWT4sK4pjZgDQetyiLvMk_WHvqq_BhAxLO8V9ZX8kVUwz47e6VwaR2A1Kr4lhnHkawqzDUNduXKnrZg2IzjNlEKkcdcc3fv1jMpzAFEc9Dd3OJBOoDjBIj/s640/R6.JPG" width="640" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
</div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "courier new";"><span style="font-size: 13.3333px;"></span></span></div>
<pre class="brush:sql">USE [WideWorldImporters]
DECLARE @SQL NVARCHAR(MAX) = N'
SELECT [OrderID]
,[CustomerID]
,[SalespersonPersonID]
,[PickedByPersonID]
,[ContactPersonID]
,[BackorderOrderID]
FROM [WideWorldImporters].[Sales].[Orders];'
DECLARE @RScript NVARCHAR(MAX) =N'
Sys.setenv(JAVA_HOME="C:/Program Files/Java/jdk-11.0.1/")
library(rJava);
library("xlsx");
write.xlsx2(InputDataSet, file = "H:/Temp/Blog/ImportExportExcel/Sales.xlsx", sheetName="Order",
col.names=TRUE, append=FALSE) '
EXEC sp_execute_external_script
@language = N'R'
,@script = @RScript
,@input_data_1 = @SQL
</pre>
</div>
<div>
</div>
<div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWOg7TC0Sqdv7cYq8UoQMzfc13ZrYDxTbo7R0G81xEzR9lm5E7nKA15_D4B-nm7dj-V0ke1Dt779LHCrbejzXBNw5V9c_chYaDHnczUG-jtvt70OrxXbyK0-JRxTCZDlUyNELCcWbf_bQj/s1600/R7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="712" data-original-width="963" height="472" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWOg7TC0Sqdv7cYq8UoQMzfc13ZrYDxTbo7R0G81xEzR9lm5E7nKA15_D4B-nm7dj-V0ke1Dt779LHCrbejzXBNw5V9c_chYaDHnczUG-jtvt70OrxXbyK0-JRxTCZDlUyNELCcWbf_bQj/s640/R7.JPG" width="640" /></a></div>
<div>
</div>
<div>
</div>
<div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><br /></i></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i>73,595</i> records got exported in <i>~10 seconds</i>. Note I have used "write.xlsx2" in above R script as it is faster than "write.xlsx".<o:p></o:p></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">More information on this topic, to be continued in <a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank"><i>part 2</i></a> ...</span><span style="font-family: "arial" , sans-serif; font-size: 13.5pt;"><o:p></o:p></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "arial" , sans-serif; font-size: 13.5pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
</div>
<div class="MsoNormal" style="line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-outline-level: 3; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<b><i><span style="font-family: "arial" , sans-serif; font-size: 13.5pt;">See Also: </span></i></b><b><span style="font-family: "arial" , sans-serif; font-size: 13.5pt;"><o:p></o:p></span></b></div>
</div>
<div>
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/2018/12/sql-server-exportimport-excel-using-r_23.html" target="_blank">SQL Server Export/Import excel using R script - Part 2</a></i></span></li>
</ul>
</div>
<div>
</div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-76253353898300800152018-02-10T15:46:00.000+05:302018-02-10T15:46:26.708+05:30How to create linked server between On-premise and Azure SQL data warehouse<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">In my previous post, l wrote about <a href="http://www.allaboutmssql.com/2018/01/how-to-create-linked-server-between-on.html" target="_blank"><i>how to create linked server between On-premise and Azure SQL database</i></a>.</span><br />
<span style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br />
</span> <span data-iceapw="37" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">In this post, let us see how to create linked server between On-premise and Azure SQL data warehouse and after establishing linked server, how we can query Azure SQL data warehouse tables from </span><span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">local On-premise SQL Server.</span><br />
<span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white; font-size: 15.4px;">I have created Azure SQL data warehouse in my <i><a href="http://www.allaboutmssql.com/2017/12/how-to-load-data-from-azure-data-lake.html" target="_blank">previous post</a> </i>and for this example I have created table "dwtable1" within azure sql dw.</span></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white; font-size: 15.4px;"><br />
</span></span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: white; font-size: 15.4px;">Below are the steps to publish linked server connection between On-premise and azure sql dw.</span></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="font-size: 15.4px;"></span></span><br />
<a name='more'></a><br />
<span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><br />
</span> <span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><span style="font-size: 15.4px;">From SSMS, connect to On-premise SQL Server. Goto </span><b data-iceapw="8" style="font-size: 15.4px;">Object Explorer -> Server Objects -> Linked servers</b></span><br />
<span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><br />
</b></span> <span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><span data-iceapw="6" style="font-size: 15.4px; font-weight: 400;">Linked Server - Linked server name. </span></b></span><span style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">I have provided the linked server name as </span><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">AZSQLDW.</span><br />
<span data-iceapw="4" style="background-color: white; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><span data-iceapw="6" style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px; font-weight: 400;">Provider - SQL native client 11.0</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-weight: 400;" /><span data-iceapw="7" style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px; font-weight: 400;">Data source - Azure SQL Server detail</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-weight: 400;" /><span data-iceapw="5" style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px; font-weight: 400;">Catalog - Azure SQL data warehouse. I have created azure sql dw with name trnazsqldw</span></b></span><br />
<span data-iceapw="4" style="background-color: white; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><span data-iceapw="5" style="font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px; font-weight: 400;"><br />
</span></b></span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyYG5vQOhs654eQeMuJFQC1ZJQdvaGbpGs-xslPAd7iE8iSus4Dhmu3zSYJMXW9QyT5fDBZCtla8jqvfp3m01YqiREsyyjv0u1NL-K319MUxU_3Hcp9C4V4FMeT2L2GYoZ9kua7TivSgj0/s1600/dw1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="624" data-original-width="693" height="576" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyYG5vQOhs654eQeMuJFQC1ZJQdvaGbpGs-xslPAd7iE8iSus4Dhmu3zSYJMXW9QyT5fDBZCtla8jqvfp3m01YqiREsyyjv0u1NL-K319MUxU_3Hcp9C4V4FMeT2L2GYoZ9kua7TivSgj0/s640/dw1.JPG" width="640" /></a></div>
<br />
<span style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;">Provide Remote login & password (Azure SQL server authentication):</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkf_uyl_gXHtABogiXmjqqhKBLgPOFKX8KThLAkoEVoeadTr4LrPmILrirxz7KHmLVWUmnVmktBHGzkYlDYVvNGf-N8mSLR8n5OnPC_z4suru7AWhNUSMeJbNWUc85BiMzXdz8kNr3QcwF/s1600/dw2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="627" data-original-width="693" height="578" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkf_uyl_gXHtABogiXmjqqhKBLgPOFKX8KThLAkoEVoeadTr4LrPmILrirxz7KHmLVWUmnVmktBHGzkYlDYVvNGf-N8mSLR8n5OnPC_z4suru7AWhNUSMeJbNWUc85BiMzXdz8kNr3QcwF/s640/dw2.JPG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhG2ZmbjodiSJAfnIxp_NmtYqV1xi20UA80HDWwwln0OhBrpzmUkscruUwZz1dH7WBK4ImTAe8VuFXe4teK0GdpgidNDyIqzPkStd27Q85PfZ5DKAVOZEZphHBBd3CeOFxutdquznNXCzsm/s1600/dw3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="629" data-original-width="691" height="582" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhG2ZmbjodiSJAfnIxp_NmtYqV1xi20UA80HDWwwln0OhBrpzmUkscruUwZz1dH7WBK4ImTAe8VuFXe4teK0GdpgidNDyIqzPkStd27Q85PfZ5DKAVOZEZphHBBd3CeOFxutdquznNXCzsm/s640/dw3.JPG" width="640" /></a></div>
<span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><br />
</b></span> <span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><span data-iceapw="21" style="font-size: 15.4px; font-weight: 400;">Now we can query the Azure SQL database from local On-premise SQL Server using four part naming convention as shown below:</span><br style="font-family: Georgia, Utopia, "Palatino Linotype", Palatino, serif; font-size: 15.4px; font-weight: 400;" /><span style="font-size: 15.4px; font-weight: 400;"><br />
</span><span style="font-family: "georgia" , "utopia" , "palatino linotype" , "palatino" , serif; font-size: 15.4px; font-weight: 400;"></span><span data-iceapw="4" style="background-color: #0b5394; color: #f3f3f3; font-size: 15.4px; font-weight: 400;">SELECT * FROM Linkedservername.databasename.schemaname.tablename</span></b></span><br />
<span data-iceapw="4" style="background-color: white; font-family: "arial" , "helvetica" , sans-serif; font-size: 15.4px;"><b data-iceapw="8" style="font-size: 15.4px;"><span data-iceapw="4" style="background-color: #0b5394; color: #f3f3f3; font-size: 15.4px; font-weight: 400;"><br />
</span></b></span> <span style="font-family: "arial" , "helvetica" , sans-serif;"><span style="background-color: #eeeeee; font-size: 15.4px;"><br />
</span></span><br />
<br />
<pre class="brush:sql">--Select data in azure sql dw from on-premise sql server using four part naming convention
SELECT * FROM AZSQLDW.[trnazsqldw].dbo.dwtable1
--Insert data into azure sql dw from on-premise sql server using EXEC command
EXEC ( 'INSERT [trnazsqldw].dbo.dwtable1 (Id)
VALUES(1)' ) AT AZSQLDW;
--Update data in azure sql dw from on-premise sql server using four part naming convention
UPDATE AZSQLDW.[trnazsqldw].dbo.dwtable1 SET Id = 100
WHERE Id = 1
--Delete data in azure sql dw from on-premise sql server using four part naming convention
DELETE FROM AZSQLDW.[trnazsqldw].dbo.dwtable1
WHERE Id = 1
</pre>
<br />
<br />
<h3 style="text-align: left;">
</h3>
<h3 style="text-align: left;">
<span style="font-size: small;"><span data-iceapw="4" style="font-family: "arial" , "helvetica" , sans-serif;"><b data-iceapw="8"><span data-iceapw="4" style="background-color: #eeeeee;">See Also:</span></b></span></span></h3>
<ul style="text-align: left;">
<li><span data-iceapw="4" style="font-family: "arial" , "helvetica" , sans-serif;"><b data-iceapw="8"><span data-iceapw="4" style="background-color: #eeeeee; font-weight: 400;"><b data-iceapw="8" style="background-color: transparent;"><span data-iceapw="4" style="font-weight: 400;"><i><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></i></span></b></span></b></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com1tag:blogger.com,1999:blog-954987329016201413.post-54376738891472576492018-02-10T15:44:00.001+05:302018-02-10T15:44:46.327+05:30How to reset azure sql admin password<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, I have shared the resource which explains different ways to reset azure sql admin password</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">1) From Azure portal - >SQL Server Overview page -> Reset Password</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">2) From Powershell -> using Azure CLI command - az sql server update </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">3) From Powershell -> using power shell command - Set-AzureRmSqlServer</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">4) From SSMS -> using t-sql command - ALTER LOGIN</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><br /></i></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://blogs.msdn.microsoft.com/azuresqldbsupport/2017/07/23/reset-lost-admin-account-password/" target="_blank"><i>Reset lost admin account password</i></a></span><br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">See Also:</span></h3>
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-55348259622390568962018-02-09T01:28:00.004+05:302018-02-09T01:32:55.232+05:30How to delete files in Azure data lake store<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how to delete files in <i><a href="http://www.allaboutmssql.com/2017/12/getting-started-with-azure-data-lake.html" target="_blank">Azure data lake store</a> </i>using powershell and Azure CLI commands.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have created Azure data lake store with account name trndls and uploaded some JSON files.</span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><br />
<h4 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u>Using Powershell:</u></span></h4>
<br />
<pre class="brush:powershell"># Log in to your Azure account
Login-AzureRmAccount
# List all the subscriptions associated to your account
Get-AzureRmSubscription
# Select a subscription
Set-AzureRmContext -SubscriptionId "subscription id"
# Azure powershell command to delete multiple files by passing the file names separated by comma in -Paths argument
Remove-AzureRmDataLakeStoreItem -AccountName "trndls" -Paths "/myfolder/Doc1.Json","/myfolder/Doc2.Json"
# Azure powershell command to delete all files inside the folder
Remove-AzureRmDataLakeStoreItem -AccountName "trndls" -Paths /myfolder -Recurse -Clean -Force
# - Recurse - deletes all items in the target folder, including subfolders.
# Unless you specify the Clean parameter, the target folder is also deleted
# - Clean - removes all of the contents of the target folder and retains the folder
# - Force - to run without asking for user confirmation
</pre>
<br />
<br />
<h4 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;"><u>Using Azure CLI:</u></span></h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can execute the Azure CLI commands from windows powershell.</span><br />
<br />
<pre class="brush:powershell"># Log in to your Azure account
az login
# Authenticate from browser - https://aka.ms/devicelogin
# Select a subscription
az account set --subscription "subscription id"
# Azure CLI command to delete all files including the folder
az dls fs delete --account trndls --path /myfolder --recurse
</pre>
<br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">See Also:</span></h3>
<div style="text-align: left;">
</div>
<ul>
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></i></li>
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2017/12/how-to-load-data-from-azure-data-lake.html" target="_blank">How to load data from Azure Data Lake Store into Azure SQL Data Warehouse</a></i></li>
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/2013/03/executing-powershell-commands-using-ssis.html" target="_blank">Executing Powershell commands using SSIS</a></i></li>
</ul>
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Reference:</span></h3>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<div>
<ul style="text-align: left;"><span style="font-family: "arial" , "helvetica" , sans-serif;"><i>
<li><a href="https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-get-started-cli-2.0" target="_blank">Get started with Azure Data Lake Store using Azure CLI 2.0</a></li>
<li><a href="https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-get-started-powershell" target="_blank">Get started with Azure Data Lake Store using Azure PowerShell</a></li>
</i></span></ul>
</div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span></div>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-25567042817850738182018-02-08T01:06:00.001+05:302018-02-08T01:06:15.604+05:30How to copy multiple tables to Azure blob using Azure data factory<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: Arial, Helvetica, sans-serif;">In my previous article, I wrote about <a href="http://www.allaboutmssql.com/2018/01/azure-data-factory-v2.html" target="_blank"><i>introduction on ADF v2</i></a>. In this post, let us see how to copy multiple tables to Azure blob using ADF v2 UI.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">For this example, I have created <a href="http://www.allaboutmssql.com/2015/12/microsoft-azure-create-sql-objects.html" target="_blank"><i>tables</i></a> named <b><i>Test, Test1</i></b> within <i><a href="http://www.allaboutmssql.com/2015/12/microsoft-azure-create-sql-server.html" target="_blank">Azure SQL database</a> - </i>Source for the copy operation.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I have created <a href="http://www.allaboutmssql.com/2017/12/azure-blob-storage-and-container.html" target="_blank"><i>Azure blob with Container</i></a> called <b><i>myfolder</i></b> - Sink for the copy operation.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">To copy multiple tables to Azure blob in JSON format, created</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><i>Pipeline1</i></b> - For Each activity with Copy activity</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><i>Pipeline2</i></b> - Lookup activity and Execute pipeline activity</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Lookup activity provides the list of tables, output of Lookup are mapped to Object type parameter. For Each activity on each iteration (tables list in object parameter) provides the table name to source & sink within copy activity.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Below are the steps:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"></span><br />
<h4 style="text-align: left;">
<a name='more'></a><br /></h4>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Create two Linked services 1) Pointing to Azure SQL database 2) Pointing to Azure blob folder</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjs1oKdEDnCQTxIpRgzbaH5pDEb2C3nPB3nuOvTJ9yx7YT5pX3ijjvJvg4oYNKCI_0_s4pfzA59v13pRloUwOa6OWIR6XMbXWNiIh5r12GN_a2XTcNmRjPH_TzAdEi3O70XuSP4MwUUqXwv/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="267" data-original-width="605" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjs1oKdEDnCQTxIpRgzbaH5pDEb2C3nPB3nuOvTJ9yx7YT5pX3ijjvJvg4oYNKCI_0_s4pfzA59v13pRloUwOa6OWIR6XMbXWNiIh5r12GN_a2XTcNmRjPH_TzAdEi3O70XuSP4MwUUqXwv/s640/1.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Create three datasets 1) for Lookup activity (Azure sql database) </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"> 2) for source (Azure sql database) </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"> 3) for sink (Azure blob folder)</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Lookup dataset - just point to the azure sql database connection:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCg7Ivy-AUQeB0be4DFsC9KZOESsLPWMtaH5A46qkGIPIi2YtPSD4JMov9lQT-zGkOD1KZXXPw2bP6VU5OO0yVkndE-a62dDH_cFi_J9WG5G78TwyGXX1Azz3nZA7NgLg2Dz1pCgscPx-a/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="487" data-original-width="785" height="396" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCg7Ivy-AUQeB0be4DFsC9KZOESsLPWMtaH5A46qkGIPIi2YtPSD4JMov9lQT-zGkOD1KZXXPw2bP6VU5OO0yVkndE-a62dDH_cFi_J9WG5G78TwyGXX1Azz3nZA7NgLg2Dz1pCgscPx-a/s640/2.JPG" width="640" /></a></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Source dataset for copy operation - just point to azure sql database connection and edit the table name as dummy:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaoWBHJl2TqFZRH2QHG5ZpcqEmr5vVcqCVOVErLVdV8GwUUG5EkGO6U2ikvFus_R-OKrXWHF2rqtoLqD53KfRwcCix6Ye8ot531M_nWLQ7yG3bP98CfcWnRk18_Cbfy1NE8EdU0i5gjZTK/s1600/3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="485" data-original-width="814" height="380" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaoWBHJl2TqFZRH2QHG5ZpcqEmr5vVcqCVOVErLVdV8GwUUG5EkGO6U2ikvFus_R-OKrXWHF2rqtoLqD53KfRwcCix6Ye8ot531M_nWLQ7yG3bP98CfcWnRk18_Cbfy1NE8EdU0i5gjZTK/s640/3.JPG" width="640" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Sink dataset for copy operation - just point to azure blob connection and edit the file name as <u>Add dynamic content</u>:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Also select the file format, for this example I have Json format.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlY_TXlygT2pfz8rkL92gSljn-Hqi-NkCRlJcj23s-GYlE_Sj7bg0xLhtOGeM5qHe2KelkjAtf_c9aWJXc57vpai1UpGQrOJGGGq0ywN1FWOipy8LME2UhDHJSQ1vd3iZ4YGy2R6I-_cO5/s1600/4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="661" data-original-width="776" height="544" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlY_TXlygT2pfz8rkL92gSljn-Hqi-NkCRlJcj23s-GYlE_Sj7bg0xLhtOGeM5qHe2KelkjAtf_c9aWJXc57vpai1UpGQrOJGGGq0ywN1FWOipy8LME2UhDHJSQ1vd3iZ4YGy2R6I-_cO5/s640/4.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Click + New to add parameter, I have created parameter with name <b><i><span style="background-color: #0b5394; color: #eeeeee;">azuresqltblname</span></i></b>.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">In the <u>Parameterizable properties</u> -> set the file name as <span style="background-color: #0b5394; color: #f3f3f3;"><b><i>@{dataset().azuresqltblname}</i></b></span></span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHzdFw_XPc4Yp2FRNRUuD_RAXw72dUBVxuAMVZP19hr-jOM9FplXWsjkv7W26CcPbE9nXFeDbNNshFDKGCqmBVlzWuBmd7yHH96oIGIvlajccAe8RZzWP4e0lv5keAj7ZuCoYctJn7flgR/s1600/5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="658" data-original-width="693" height="606" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHzdFw_XPc4Yp2FRNRUuD_RAXw72dUBVxuAMVZP19hr-jOM9FplXWsjkv7W26CcPbE9nXFeDbNNshFDKGCqmBVlzWuBmd7yHH96oIGIvlajccAe8RZzWP4e0lv5keAj7ZuCoYctJn7flgR/s640/5.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Create a pipeline - pipeline 1 and then click on the empty design window -> Parameters to create <u>object parameter</u> named <b><i><span style="background-color: #0b5394; color: #f3f3f3;">tableslist</span></i></b></span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPVnPyZ5fl5cwtudwonThN0UbChJi1ZMUkRauPjTRRDq2_pk3bm1RZn4emC1nM_arI-aGxelRf6K6EoIZA6tqivOLewFDPBNSZTQMGWq0S0e_m-RTrRhjPfziEn8G7DlinxEwvryUt76oE/s1600/6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="265" data-original-width="692" height="244" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPVnPyZ5fl5cwtudwonThN0UbChJi1ZMUkRauPjTRRDq2_pk3bm1RZn4emC1nM_arI-aGxelRf6K6EoIZA6tqivOLewFDPBNSZTQMGWq0S0e_m-RTrRhjPfziEn8G7DlinxEwvryUt76oE/s640/6.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Now drag&drop ForEach activity and Settings -> select <u>sequential</u> and provide the items as</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="color: #f3f3f3; font-family: Arial, Helvetica, sans-serif;"><b><i style="background-color: #0b5394;">@pipeline().parameters.tableslist</i></b></span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaUTQBecTGJ9RTXfcyPEAKfejLwhwaZSeOtkRw7SLFaUpsv1hcn1H4UwAkOPjFJ8KTVIUKRqohDhJCvyubHksRLB3CPONtxL3onn2MQHdnPUXkIZWhFAyFx_COOGODtRR27D9nFmdVwenR/s1600/7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="815" height="502" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaUTQBecTGJ9RTXfcyPEAKfejLwhwaZSeOtkRw7SLFaUpsv1hcn1H4UwAkOPjFJ8KTVIUKRqohDhJCvyubHksRLB3CPONtxL3onn2MQHdnPUXkIZWhFAyFx_COOGODtRR27D9nFmdVwenR/s640/7.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Click on Activities -> Edit Activities -> drag&drop Copy activity </span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Select the source dataset and query as <span style="background-color: #0b5394; color: #f3f3f3;"><b><i>SELECT * FROM @{item().name}</i></b></span></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">(which means name from lookup query is passed to object parameter and on each iteration table name is passed dynamically)</span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7SZJnmIkZ7evuYzCYgmQ7l6PgQAOZZ46w22lTyZ8WyBAi2_y7VQV1weFQ2Mp_jKXngCOa_ph02v0Ove34vD3fVdRdBfX6WqM_1vH2yDSvstNoB0uhQem-pf0THRdMZd4qIJEOylqiYJ2Z/s1600/8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="660" data-original-width="789" height="534" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7SZJnmIkZ7evuYzCYgmQ7l6PgQAOZZ46w22lTyZ8WyBAi2_y7VQV1weFQ2Mp_jKXngCOa_ph02v0Ove34vD3fVdRdBfX6WqM_1vH2yDSvstNoB0uhQem-pf0THRdMZd4qIJEOylqiYJ2Z/s640/8.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Select the sink dataset:</span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh19WGyjFmhWzGlvrYGg93UX1jdErtr1dc4x81S9sk4OXc1uVMscsq_bWsXhAaxjGkFvHSI3QYvhO32i8P8f2U6eqZyRTcvveW5aylQO7kALMZf1AMKJAC7Ji8jtNB1Jmsxx8PzSkkxIf0g/s1600/9.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="191" data-original-width="750" height="162" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh19WGyjFmhWzGlvrYGg93UX1jdErtr1dc4x81S9sk4OXc1uVMscsq_bWsXhAaxjGkFvHSI3QYvhO32i8P8f2U6eqZyRTcvveW5aylQO7kALMZf1AMKJAC7Ji8jtNB1Jmsxx8PzSkkxIf0g/s640/9.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">In the Parameters tab -> Provide value for the dynamic property (file name) set in the sink dataset as <b><i><span style="background-color: #0b5394; color: #f3f3f3;">@{item().name}</span></i></b></span></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">(which means name from lookup query is passed to object parameter and on each iteration table name is passed dynamically)</span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIh8epIh3EaQBFXQlkgK5Psk1nQIntqdVQyKHdcI8WePW5p__PyaxrlO9ig8IweWE9NnNL8zZVUDgjUIxxPNt5Z8CEi3JCQ5mmuId5UWEWrK9eO5smVc6SROOIq3fyvp69y72RCFpRt3lc/s1600/10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="293" data-original-width="537" height="348" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIh8epIh3EaQBFXQlkgK5Psk1nQIntqdVQyKHdcI8WePW5p__PyaxrlO9ig8IweWE9NnNL8zZVUDgjUIxxPNt5Z8CEi3JCQ5mmuId5UWEWrK9eO5smVc6SROOIq3fyvp69y72RCFpRt3lc/s640/10.JPG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Create pipeline - <b><i>Pipeline2.</i></b> Drag&drop Lookup and Execute Pipeline activity as shown below:</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">In the Lookup settings, select the dataset and query to return list of table names</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both;">
<span style="background-color: #0b5394; color: #f3f3f3; font-family: Arial, Helvetica, sans-serif;">SELECT name FROM sys.tables</span></div>
<div class="separator" style="clear: both;">
<span style="background-color: #0b5394; color: #f3f3f3; font-family: Arial, Helvetica, sans-serif;">WHERE name LIKE '%test%'</span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both;">
<span style="font-family: Arial, Helvetica, sans-serif;">Also uncheck the <u>First row only option</u></span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGICIbpj6IGArNzFhqfOYXH_ffnpkS9vdMnMM_CD4LCm7OTPuwrnt7iYLSiz0xtZJQkafxaIxSl4OoFC2a1X1Ms-syEiwk2oy7JnHqIZN6uF3hmiLiIlW58iKSDnXwlmWLu-f6BheATUge/s1600/11.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="585" data-original-width="755" height="494" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGICIbpj6IGArNzFhqfOYXH_ffnpkS9vdMnMM_CD4LCm7OTPuwrnt7iYLSiz0xtZJQkafxaIxSl4OoFC2a1X1Ms-syEiwk2oy7JnHqIZN6uF3hmiLiIlW58iKSDnXwlmWLu-f6BheATUge/s640/11.JPG" width="640" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In the Execute pipeline settings -> Invoke pipeline 1 and create input parameter for pipeline1 </span><span style="font-family: Arial, Helvetica, sans-serif;">and then map the output of lookup to this parameter as</span><br />
<span style="background-color: #0b5394; color: #f3f3f3; font-family: Arial, Helvetica, sans-serif;"><b><i>@activity('LookupTablesList').output.value</i></b></span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_Ow4ZeFXWpHHvcI6ODylT5qV_DyPk122yId7NJ-drKLfuP_sbs-gc5CVU_vgEw16EjVqPPUPF9uEMkXm4ZEjymSSNrUV1p_bL3TIg6p8KOt-hLxTRZGg17hxHC7tQazaZjrwR4SV6_Lyc/s1600/12.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="653" data-original-width="641" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_Ow4ZeFXWpHHvcI6ODylT5qV_DyPk122yId7NJ-drKLfuP_sbs-gc5CVU_vgEw16EjVqPPUPF9uEMkXm4ZEjymSSNrUV1p_bL3TIg6p8KOt-hLxTRZGg17hxHC7tQazaZjrwR4SV6_Lyc/s640/12.JPG" width="628" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">Now publish the changes, validate the pipeline and trigger pipeline2 and look into the Monitor window:</span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEzeg8xxBx0HPo3pW9j8Ndu8QZCimh9BDZDX6mir9Vv9Q_xqKwVCaPn5TIVfMsKTRZcEroGPD_vH6TIfQFjg6BTWw1k8DcaCkMM3zWnhhw8C4PW1iBQQ1WRrOEd65A0xR2c1Hxr2MTpFVF/s1600/Last.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="289" data-original-width="1352" height="136" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEzeg8xxBx0HPo3pW9j8Ndu8QZCimh9BDZDX6mir9Vv9Q_xqKwVCaPn5TIVfMsKTRZcEroGPD_vH6TIfQFjg6BTWw1k8DcaCkMM3zWnhhw8C4PW1iBQQ1WRrOEd65A0xR2c1Hxr2MTpFVF/s640/Last.JPG" width="640" /></a></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">All the tables in Azure sql database with name like '%test%' will be copied as JSON file into Azure blob folder.</span><br />
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">See Also:</span></h3>
<br />
<ul style="text-align: left;">
<li><i style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></i></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-17126397898591080582018-01-29T02:34:00.003+05:302018-01-29T02:38:57.009+05:30SQL Server Row level security<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In SQL Server, implementing Row-Level Security enables to control access to rows in a database table based on the </span><span style="font-family: "arial" , "helvetica" , sans-serif;">characteristics of the user executing a query (e.g., group membership or execution context). </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see an example for RLS with Session_Context, advantage of this technique is we don't need to create separate users to enforce row level security.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">We can implement RLS by creating SECURITY POLICY </span><span style="font-family: "arial" , "helvetica" , sans-serif;">and predicates created as inline table valued functions. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Let us consider this use case, an Insurance provider company database has all the agency company, its adviser, policy and commission</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> details. Insurance provider has the application dashboard where any agency / individual adviser from an agency company can login (using single sign on) and view their details.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Requirement for Insurance provider is one agency company data should not be exposed to other company and only admins can modify the commission data within their company.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;">Using RLS and by following industry standard data model for Insurance & User role access,</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">above requirement can be handled effectively.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-D7nN3pghHu1WToX5aBTVOMqmQsriwfCeoAPvRrYsTc2aLm-gruzlxvGjq54XE5cKwPP4_3GMqiuNopOyQ89p0v8CP4Coa5UH0b7SAAh6sJmn5LleZqDGnNA_A1PHgX5moCBo-9OdAjaT/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="514" data-original-width="825" height="398" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-D7nN3pghHu1WToX5aBTVOMqmQsriwfCeoAPvRrYsTc2aLm-gruzlxvGjq54XE5cKwPP4_3GMqiuNopOyQ89p0v8CP4Coa5UH0b7SAAh6sJmn5LleZqDGnNA_A1PHgX5moCBo-9OdAjaT/s640/1.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Below is the sample data script:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<br />
<br />
<br />
<br />
<pre class="brush:sql">DROP DATABASE IF EXISTS InsuranceProvider
CREATE DATABASE InsuranceProvider
GO
USE InsuranceProvider
GO
CREATE TABLE Company(Id INT,Name VARCHAR(100),
CONSTRAINT PK_Company PRIMARY KEY (Id))
INSERT Company SELECT 1,'company 1'
INSERT Company SELECT 2,'company 2'
--SSO user table, during user registration all user information like username, password will be stored in this table
--For this example, I have this trimmed version of table. User in this example are referred as Agents
CREATE TABLE [User] (Id BIGINT IDENTITY(1,1),Name VARCHAR(100),
CONSTRAINT PK_User PRIMARY KEY(Id))
INSERT [User] (Name) SELECT 'Agency company 1'
INSERT [User] (Name) SELECT 'Advisor - Agency 1'
INSERT [User] (Name) SELECT 'Advisor - Agency 1'
INSERT [User] (Name) SELECT 'Agency company 2'
INSERT [User] (Name) SELECT 'Advisor - Agency 2'
INSERT [User] (Name) SELECT 'Advisor - Agency 2'
--Relationship between party and party members relationship (Insurance industry standard data model)
CREATE TABLE UserCompanyRelationship(UserId BIGINT,ParentId BIGINT, CompanyId INT,
CONSTRAINT PK_UserCompanyRelationship PRIMARY KEY(UserId,CompanyId),
CONSTRAINT FK_UserCompanyRelationship_User FOREIGN KEY (UserId)
REFERENCES [User] (Id),
CONSTRAINT FK_UserCompanyRelationship_Company FOREIGN KEY (CompanyId)
REFERENCES Company (Id)
)
INSERT UserCompanyRelationship SELECT 1,NULL,1
INSERT UserCompanyRelationship SELECT 2,1,1
INSERT UserCompanyRelationship SELECT 3,1,1
INSERT UserCompanyRelationship SELECT 4,NULL,2
INSERT UserCompanyRelationship SELECT 5,2,2
INSERT UserCompanyRelationship SELECT 6,2,2
CREATE TABLE Permission(Id INT,Name VARCHAR(100),
CONSTRAINT PK_Permission PRIMARY KEY (Id))
INSERT Permission SELECT 1,'Agency super User'
INSERT Permission SELECT 2,'Advisor normal User'
CREATE TABLE UserPermission (Id BIGINT IDENTITY(1,1),UserId BIGINT,PermissionId INT,
CONSTRAINT PK_UserPermission PRIMARY KEY (Id),
CONSTRAINT FK_UserPermission_User FOREIGN KEY (UserId)
REFERENCES [User] (Id),
CONSTRAINT FK_PK_UserPermission_Permission FOREIGN KEY (PermissionId)
REFERENCES Permission (Id))
INSERT UserPermission SELECT 1,1
INSERT UserPermission SELECT 2,2
INSERT UserPermission SELECT 3,2
INSERT UserPermission SELECT 4,1
INSERT UserPermission SELECT 5,2
INSERT UserPermission SELECT 6,2
CREATE TABLE Commission (Id BIGINT IDENTITY(1,1),UserId BIGINT, Amount NUMERIC(18,0),
CONSTRAINT FK_Commission_User FOREIGN KEY (UserId)
REFERENCES [User] (Id))
INSERT Commission SELECT 1,300
INSERT Commission SELECT 1,50
INSERT Commission SELECT 2,100
INSERT Commission SELECT 3,100
INSERT Commission SELECT 4,300
INSERT Commission SELECT 4,75
INSERT Commission SELECT 5,100
INSERT Commission SELECT 6,100
</pre>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Filter predicates silently filter the rows available to read operations (SELECT, UPDATE, and DELETE). </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Block predicates explicitly block write operations (AFTER INSERT, AFTER UPDATE, BEFORE UPDATE, BEFORE DELETE) that violate the predicate.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Below is the script to create inline predicate function and security policy:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<br />
<br />
<br />
<pre class="brush:sql">DROP SECURITY POLICY IF EXISTS Security.AgentCommissionAccessSecurityPolicy
DROP FUNCTION IF EXISTS Security.AgentCommissionAccessPredicate
GO
DROP SCHEMA IF EXISTS Security
GO
CREATE SCHEMA Security
GO
--predicate function
CREATE FUNCTION Security.AgentCommissionAccessPredicate(@UserId INT)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
SELECT 1 AS CommAccess
FROM dbo.UserCompanyRelationship UCR
WHERE UCR.CompanyId = CAST(SESSION_CONTEXT(N'CompanyId') AS INT)
AND UCR.UserId = @UserId
GO
--Security policy to enforce RLS
CREATE SECURITY POLICY Security.AgentCommissionAccessSecurityPolicy
ADD FILTER PREDICATE Security.AgentCommissionAccessPredicate(UserId) ON dbo.Commission
,ADD BLOCK PREDICATE Security.AgentCommissionAccessPredicate(UserId) ON dbo.Commission
GO
</pre>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below is the script to test, open two or three sessions in SSMS, pass different userId's</span><br />
<br />
<br />
<br />
<pre class="brush:sql">--Below is the test script, we can pass userid from a company
--Test - one company cannot view other company data
--Consider UserId is captured through single sign on and application passes the userId to backend
DECLARE @UserId INT = 5, -- input
@CompanyId INT,
@ParentId INT
--Below code is to capture the companyid based on the userid and set the session context
--so that people from one company cannot see other company data
--another logic here is capturing the parentid, in case of super user login, they can view all the data
SELECT @CompanyId = UCR.CompanyId, @UserId = CASE WHEN UCR.ParentId IS NULL THEN NULL ELSE UCR.UserId END
FROM UserCompanyRelationship UCR
WHERE UCR.UserId = @UserId
EXEC sp_set_session_context N'CompanyId',@CompanyId
SELECT * FROM Commission C
WHERE UserId = @UserId
OR @UserId IS NULL
</pre>
<br />
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Below is the script to test, open two or three sessions in SSMS, pass different userId's</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<br />
<br />
<br />
<pre class="brush:sql">--Below is the test script, we can pass userid from a company
--Test - only user with admin privilege can insert/update/delete commission data within their company
--Consider UserId is captured through single sign on and application passes the userId to backend
DECLARE @UserId INT = 1, -- input
@UpdateUser INT = 6, -- agent for whom commission amount needs to be modified
@CompanyId INT,
@PermissionId INT
SELECT @CompanyId = UCR.CompanyId, @PermissionId = UP.PermissionId
FROM UserCompanyRelationship UCR
JOIN UserPermission UP
ON UP.UserId = UCR.UserId
WHERE UCR.UserId = @UserId
EXEC sp_set_session_context N'CompanyId',@CompanyId
--Only admin user can insert/update/delete a record in commission table
IF @PermissionId = 1
BEGIN
UPDATE Commission SET Amount = -10
WHERE UserId = @UpdateUser
END
</pre>
<br />
<br />
<h3 style="text-align: left;">
</h3>
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Reference:</span></h3>
<div>
<ul style="text-align: left;">
<li><i style="font-weight: normal;"><span style="font-family: Arial, Helvetica, sans-serif; font-size: small;"><a href="https://docs.microsoft.com/en-us/sql/relational-databases/security/row-level-security" target="_blank">Row-level security</a></span></i></li>
</ul>
</div>
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">See Also:</span></h3>
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/p/t-sql-scripts_19.html" target="_blank">My posts on T-SQL</a></i></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com1tag:blogger.com,1999:blog-954987329016201413.post-71769890694413502772018-01-23T03:18:00.002+05:302018-01-23T03:23:08.319+05:30Temp tables in Azure sql database and Azure sql data warehouse<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see about temporary tables in Azure SQL database and Azure SQL data warehouse.</span><br />
<h4 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temp tables in Azure SQL database:</span></h4>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Global (DB scoped)</b> & Local temporary tables are supported for Azure SQL Database and follow the same syntax and semantics that SQL Server uses.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Local temporary table (#table_name), </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Database scoped global temporary table (##table_name)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lets see an example for below statement:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><b>"Azure SQL Database supports global temporary tables and it can be accessed only by all users sessions within the same Azure SQL database."</b></i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have created two Azure sql databases azsqldb1 & azsqldb2.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As shown in below image, created temp table inside azsqldb1 from session 1</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigY48J-JJT9OknV5GYEBGxFl1ax_T2UsfzfS0c4KMU80alR4U59iW59pw3JgwOSjgK1sN7WMef5ievdBedzYwPXrf4I_DxASc0FIjXlWGZ3B_Hcdr2m1oSc8iaYB7wNImFfRcweHqjKxR6/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="320" data-original-width="800" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigY48J-JJT9OknV5GYEBGxFl1ax_T2UsfzfS0c4KMU80alR4U59iW59pw3JgwOSjgK1sN7WMef5ievdBedzYwPXrf4I_DxASc0FIjXlWGZ3B_Hcdr2m1oSc8iaYB7wNImFfRcweHqjKxR6/s640/1.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">As shown in below image, temp table was accessible from session 2, this session points to database azsqldb1 where temp table was created.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgj72_Xsldk6PSMlq9lFNPgKk_QFhQPuvnR6akIH-q5DUgEYjr5cvnoPNj6AGuzxnLYwaVx_lJ1tXG1ts3DOYq3yB_xkDGaaylHGADSM239tIBCW3Lgoep3EJS09rd-FZ4kJrFXFskkrfUV/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="408" data-original-width="532" height="490" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgj72_Xsldk6PSMlq9lFNPgKk_QFhQPuvnR6akIH-q5DUgEYjr5cvnoPNj6AGuzxnLYwaVx_lJ1tXG1ts3DOYq3yB_xkDGaaylHGADSM239tIBCW3Lgoep3EJS09rd-FZ4kJrFXFskkrfUV/s640/2.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">As shown in below image, temp table was not accessible when queried from another session which points to database azsqldb2.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyoVCNDNIBK20e0suJ7XFSwQ9vdjDE6lSpEXpFVZcgHdpXk1rDIdK0_SQXWcjeTFu9ehJ24JVaKd0WkTBtyr0zT3IR4lrhN56AlQDqCd7SkmD4lOCOr3urIvgH2AzdyLlhJxX5K91vpiY7/s1600/3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="383" data-original-width="291" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhyoVCNDNIBK20e0suJ7XFSwQ9vdjDE6lSpEXpFVZcgHdpXk1rDIdK0_SQXWcjeTFu9ehJ24JVaKd0WkTBtyr0zT3IR4lrhN56AlQDqCd7SkmD4lOCOr3urIvgH2AzdyLlhJxX5K91vpiY7/s640/3.JPG" width="486" /></a></div>
<br />
<br />
<h4 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temp tables in Azure SQL data warehouse:</span></h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lets see an example for below statement:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><b>"In SQL Data Warehouse, temporary tables scope is at the session level when temp table is created as stand alone statement. When created inside a stored procedure it can be accessed in other sessions as well.</b></i></span><br />
<i><b><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temporary tables are slightly different in Azure SQL Data Warehouse as they can be accessed from anywhere including both inside and outside of a stored procedure."</span></b></i><br />
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have created two Azure sql DW databases DW1 & DW2.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Created a stored procedure within DW1 from session1, temp table is accessible even outside of stored procedure as shown below:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjgEphxqboTd7g-m33v5qPZvCSaPETIQgfmVOh6UdR3_Z2NioRHEWLvQGk18ZZgK670XaQ5IAT83ihuyf50Wv2KxloefaDa-jwyIYemaxtrBJAbZsn-KuZp7e7-9pccRM7f0wUgxO4YDOp/s1600/d1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="401" data-original-width="1084" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjgEphxqboTd7g-m33v5qPZvCSaPETIQgfmVOh6UdR3_Z2NioRHEWLvQGk18ZZgK670XaQ5IAT83ihuyf50Wv2KxloefaDa-jwyIYemaxtrBJAbZsn-KuZp7e7-9pccRM7f0wUgxO4YDOp/s640/d1.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temp table is accessible from session 2 (same database DW1) as shown below:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Reason behind this is, as I have executed stored procedure that creates temp table it is available in session 2.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgipzOLx6yjx0z4BrKAdKBmvjkAq8-XzAufTHeKkyjOU7Dpdv9t7rUi4iglEhx6bZx_aomptHiPKanE25-B-qhyphenhyphennwpbpI889o0x1Kv_cognMT8Xdyb1wE8CTYB4rTfVwg8T3Nc6eho2rN7B/s1600/d2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="508" data-original-width="813" height="398" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgipzOLx6yjx0z4BrKAdKBmvjkAq8-XzAufTHeKkyjOU7Dpdv9t7rUi4iglEhx6bZx_aomptHiPKanE25-B-qhyphenhyphennwpbpI889o0x1Kv_cognMT8Xdyb1wE8CTYB4rTfVwg8T3Nc6eho2rN7B/s640/d2.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temp table is not accessible from session 3 (same database DW1) as shown below:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">(As temp table scope is at session level in azure sql data warehouse)</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKN_KYqAM3cBsTjIK2HvdefcPcvgRy8hF5ZW_M-7i1HfVikpu0uRhRng0zH9MFkj-vurDh_oWJd-MUe8HLU6rERvUwnuskbwA0MkMmkt3l14AlmAuIo9MRq1V2lrQfPyiGDDq3DsesWJdq/s1600/d3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="381" data-original-width="523" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKN_KYqAM3cBsTjIK2HvdefcPcvgRy8hF5ZW_M-7i1HfVikpu0uRhRng0zH9MFkj-vurDh_oWJd-MUe8HLU6rERvUwnuskbwA0MkMmkt3l14AlmAuIo9MRq1V2lrQfPyiGDDq3DsesWJdq/s640/d3.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temp table is not accessible from session 3 (different database DW2) as shown below:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">(As global temp table is not supported in azure sql data warehouse)</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAo8rU06IeamT5vvq8eafeDo_tNZFCEocCGa788JbHcUiCwL0E978ae5WrY5w_vcy3YW-f8588bxP48yk9Ldm53P6y3zDhXWI59F0ilcrqUh76iU2ozQmIonM3Bj6XDbv8UnGh2QrbZRjq/s1600/d4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="397" data-original-width="545" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAo8rU06IeamT5vvq8eafeDo_tNZFCEocCGa788JbHcUiCwL0E978ae5WrY5w_vcy3YW-f8588bxP48yk9Ldm53P6y3zDhXWI59F0ilcrqUh76iU2ozQmIonM3Bj6XDbv8UnGh2QrbZRjq/s640/d4.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Temporary tables can also be created with a CTAS (CREATE TABLE AS SELECT)</span><br />
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Temporary tables offer a performance benefit because their results are written to local rather than remote storage.</span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif;">Global Temporary Tables are not supported.</span></li>
</ul>
<div>
<span style="font-family: arial, helvetica, sans-serif;">Initially global temp table was not supported in Azure SQL database, later it was introduced. So we might expect some changes related to temp tables in Azure sql database and Azure sql data warehouse.</span></div>
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">Reference: </span></h3>
<ul style="text-align: left;">
<li><a href="https://docs.microsoft.com/en-us/sql/t-sql/statements/create-table-transact-sql#database-scoped-global-temporary-tables-azure-sql-database" target="_blank"><i><span style="font-family: "arial" , "helvetica" , sans-serif;">Database scoped global temporary tables (Azure SQL Database)</span></i></a></li>
<li><i><span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://docs.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-tables-temporary" target="_blank">Temporary tables in SQL Data Warehouse</a></span></i></li>
</ul>
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small;">See Also: </span></h3>
<ul style="text-align: left;">
<li><a href="http://www.allaboutmssql.com/p/c.html" target="_blank"><i><span style="font-family: "arial" , "helvetica" , sans-serif;">My posts on Microsoft Azure</span></i></a></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-6606147714717906172018-01-20T02:22:00.004+05:302018-01-20T02:47:24.087+05:30Azure data factory v2<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Azure Data Factory visual tools enabled in public preview few days back. In this post, let us see how the look and feel is and whats new in ADF v2.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">From Azure portal, while creating Azure data factory we need to select the version as <b>v2</b>, once created click on <b>Author & Monitor</b>.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtOHIXyRFu5x8_smrfhUvAai7wZCMQEzjHJ7BMtpZyZBJWRRgRuX8ji6-oohD9dHvANmoDrAhIPt3E8m41PKbdXuTDm0krh9PeX9wlTckjrZ8m8v79OVK8npZwOErF9c5pYzq5oNXFAgkn/s1600/1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="662" data-original-width="1366" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtOHIXyRFu5x8_smrfhUvAai7wZCMQEzjHJ7BMtpZyZBJWRRgRuX8ji6-oohD9dHvANmoDrAhIPt3E8m41PKbdXuTDm0krh9PeX9wlTckjrZ8m8v79OVK8npZwOErF9c5pYzq5oNXFAgkn/s640/1.JPG" width="640" /></a><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In my previous post, I had shared an example to <i><a href="http://www.allaboutmssql.com/2017/12/copy-data-from-azure-blob-storage-to.html" target="_blank">copy data from Azure blob to Azure cosmos DB using Copy data wizard</a>.</i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><br /></i></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Let us see an example to copy data from Azure blob to Azure sql database using new UI.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Click on <b>Create pipeline</b> in the landing page as shown above:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If we click on + near Search Resources, we can create Pipelines, Datasets.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can create Linked Service & Dataset before or while creating pipeline.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Linked service can be created in below flow:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Connections (see in the bottom of below image) -> Click New -> Select Source or Sink -> Test Connection -> Save</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfDN4ey1newr5smMeJGHNe4qOhIUCYu54-bUuhSCxYSGefzYdlGJQKQyhSuOJqvBCSshTkdmE4ExFJkWrzb7BaWZNnuelQQbOvtLhFpTl1Tjzo6pcUU3eZsfYcVGFuKy2Ihyphenhyphen0sH_P2gNAD/s1600/1.1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="655" data-original-width="1349" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfDN4ey1newr5smMeJGHNe4qOhIUCYu54-bUuhSCxYSGefzYdlGJQKQyhSuOJqvBCSshTkdmE4ExFJkWrzb7BaWZNnuelQQbOvtLhFpTl1Tjzo6pcUU3eZsfYcVGFuKy2Ihyphenhyphen0sH_P2gNAD/s640/1.1.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Once we create a Pipeline, we can see the list of Activities that can be dragged & dropped on to canvas.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizrbygu-i7v3318lQeiS0k4KFewAQF0R-F_YA2kL2U13qFlSB3gop2e2sxTP1TPF4ivuz_SDfEBCcpdETmdgFEfLDmGKdbVDSEE0V6Fv5w1zGjIq_gA8JQQEtA4k5Zy1HP3i8sYklgd1wK/s1600/1.2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="638" data-original-width="1057" height="386" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizrbygu-i7v3318lQeiS0k4KFewAQF0R-F_YA2kL2U13qFlSB3gop2e2sxTP1TPF4ivuz_SDfEBCcpdETmdgFEfLDmGKdbVDSEE0V6Fv5w1zGjIq_gA8JQQEtA4k5Zy1HP3i8sYklgd1wK/s640/1.2.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Source dataset can be created in below flow:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Pipeline -> Click Activity -> in the bottom -> Source Dataset -> New -> Connection </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">-> Select the Source Linked service (or Click + to create new linked service) -> in the left Pencil Icon (Edit) -> Publish</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Note: We cannot close the dataset window without publishing the changes.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For this example, I have created <a href="http://www.allaboutmssql.com/2017/12/azure-blob-storage-and-container.html" target="_blank"><i>Azure blob storage and created a container</i></a> called myfolder and uploaded some files.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As shown in below image, we have an option to Test connection, Preview data & Detect format while creating dataset.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Next click on Schema to Import the schema's and publish the changes.</span><br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitnn1QNOtRJ4r5mGR7WGyFVcszXYBQFSjNUDPtvxPFre0nVh2N_6a4OknaTGC5migJUE22W5zXPTSbYySiZSoRcddYwbE0tAI242ApBh3T4FSfAT379j4-MePli2_Q8KVqsUgQM8DU9jjD/s1600/2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="659" data-original-width="1079" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitnn1QNOtRJ4r5mGR7WGyFVcszXYBQFSjNUDPtvxPFre0nVh2N_6a4OknaTGC5migJUE22W5zXPTSbYySiZSoRcddYwbE0tAI242ApBh3T4FSfAT379j4-MePli2_Q8KVqsUgQM8DU9jjD/s640/2.JPG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">After creating the source dataset, again click on pipeline window and then in the bottom click on Sink to create the target dataset.</span></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sink dataset can be created in below flow:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Pipeline -> Click Activity -> in the bottom -> Sink Dataset -> New -> Connection </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">-> Select the Sink Linked service (or Click + to create new linked service) -> in the left Pencil Icon (Edit) -> Publish</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">For this example, I have created a <a href="http://www.allaboutmssql.com/2015/12/microsoft-azure-create-sql-objects.html" target="_blank"><i>table</i></a> within <a href="http://www.allaboutmssql.com/2015/12/microsoft-azure-create-sql-server.html" target="_blank"><i>Azure SQL database</i></a>.</span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-IH7qnHT5fKGYQaEJRxpcL_ZH0k2z_xjN4dGE6SdvSExpU-LEqiUMYswO8uYbsK6L80Bxr1JMJJDLDzV5R_n4thdNMXgYZkMi7UCLl9FLo07DiDp0TXvSaB8-rh7zVuYWiWJRXgEtLDiR/s1600/3.2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="555" data-original-width="1341" height="264" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-IH7qnHT5fKGYQaEJRxpcL_ZH0k2z_xjN4dGE6SdvSExpU-LEqiUMYswO8uYbsK6L80Bxr1JMJJDLDzV5R_n4thdNMXgYZkMi7UCLl9FLo07DiDp0TXvSaB8-rh7zVuYWiWJRXgEtLDiR/s640/3.2.JPG" width="640" /></a></div>
<br />
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Next click on Schema to Import the schema's and publish the changes.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can also create parameters and write <a href="https://docs.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions" target="_blank"><i>expressions</i></a> if we have to handle some dynamic scenarios within pipeline.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqdc12vuRA1IzEbAVWM3a0BeKTyGt7-Uv6IVcsKXDAmCCiQZ-q5wHBup6dxKdPBUUGa99M_a4DXMBO0rdqwuV5vWu7U_gjGe_smSh_e2m85r0p81rRFHfAMlqOGAt0SeAxYlfcFsE6exFn/s1600/3.3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="556" data-original-width="1337" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqdc12vuRA1IzEbAVWM3a0BeKTyGt7-Uv6IVcsKXDAmCCiQZ-q5wHBup6dxKdPBUUGa99M_a4DXMBO0rdqwuV5vWu7U_gjGe_smSh_e2m85r0p81rRFHfAMlqOGAt0SeAxYlfcFsE6exFn/s640/3.3.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">After publishing the sink dataset changes, Click on Pipeline window -> Mapping (like we do the mappings between source and destination in SSIS packages)</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj50nezg3UgSDOD-b-WixhpDseO92HtlEyk1WK7ei7B3WtXTrRc3EG6VGryBgrM8oQ9lCu7bzp4KnwsrgsLP6BatNUkeR3t0NVANkW-HluTzXQoGBuSVqT1ns4E4MGd5GCBnumvoVL-Yutw/s1600/3.4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="658" data-original-width="1339" height="314" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj50nezg3UgSDOD-b-WixhpDseO92HtlEyk1WK7ei7B3WtXTrRc3EG6VGryBgrM8oQ9lCu7bzp4KnwsrgsLP6BatNUkeR3t0NVANkW-HluTzXQoGBuSVqT1ns4E4MGd5GCBnumvoVL-Yutw/s640/3.4.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">In the Settings, we have an option to keep an intermediate staging while copying the data as shown below:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAFW2uS76lbPqz154TW0KDvTcNByOZLhAgDzTAgsf8d_xe9OmIDBeu1CkVrrgvLeYBx7JiFACbIw9KwExGH0RrVthLqlepkC1Bto5-w70p-MPq6XIIDOk5vEeDBdATcn9W5E8d2Lfxhr9R/s1600/3.5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="660" data-original-width="1304" height="322" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAFW2uS76lbPqz154TW0KDvTcNByOZLhAgDzTAgsf8d_xe9OmIDBeu1CkVrrgvLeYBx7JiFACbIw9KwExGH0RrVthLqlepkC1Bto5-w70p-MPq6XIIDOk5vEeDBdATcn9W5E8d2Lfxhr9R/s640/3.5.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can now validate the entire pipeline, to see if there are any problems in pipeline / dataset.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRBdLYFq6WbkNOZ6HTV0OMv2I8TZtNjWq_YwXu9QyqRvnrX7JFMaRgxEAWEd2NY8eEBrivzVvJA5i2cbgM6K_tphIDLo-59hyB2W6kG7fnlBOP8tp2DCkgCGvuit4aD7Gr_AkODxJCozNX/s1600/3.6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="538" data-original-width="1342" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRBdLYFq6WbkNOZ6HTV0OMv2I8TZtNjWq_YwXu9QyqRvnrX7JFMaRgxEAWEd2NY8eEBrivzVvJA5i2cbgM6K_tphIDLo-59hyB2W6kG7fnlBOP8tp2DCkgCGvuit4aD7Gr_AkODxJCozNX/s640/3.6.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can do a Test Run / Trigger the pipeline. In the output window -> Actions -> we can get the execution info.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQTmLUvHwjxgT51Hu5OJmCgP9l_qzREo64xWqVcNSzHcSNt5CdF94nh89EgAmXLJCw5fSnyNO8Iwlp2hwNhwj7Aq2msE24OZoI_za5alvew1Swsq6i8pTW3y1NnDORwCxc9Mn_fgX7Wn4W/s1600/3.7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="654" data-original-width="1338" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQTmLUvHwjxgT51Hu5OJmCgP9l_qzREo64xWqVcNSzHcSNt5CdF94nh89EgAmXLJCw5fSnyNO8Iwlp2hwNhwj7Aq2msE24OZoI_za5alvew1Swsq6i8pTW3y1NnDORwCxc9Mn_fgX7Wn4W/s640/3.7.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We have successfully copied the data from blob to azure sql table using new ADF UI.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As shown in below image, we can create control flow between Activities using On Success / Failure / Completion branching.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEggzAfWtIqZHNoZno96vtlcs5m2gvd0WHZVs9sSEXzX9bam-js3r80vb25MGt1cmxzOsf3WRUFyGHFb2kJW60aGWwDIpVqGhtcGcVflRghB1eoFekaSgD8_fVxcx6xip6d3YdfpPDijo8/s1600/3.1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="414" data-original-width="887" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEggzAfWtIqZHNoZno96vtlcs5m2gvd0WHZVs9sSEXzX9bam-js3r80vb25MGt1cmxzOsf3WRUFyGHFb2kJW60aGWwDIpVqGhtcGcVflRghB1eoFekaSgD8_fVxcx6xip6d3YdfpPDijo8/s640/3.1.JPG" width="640" /></a></div>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">In the monitor page, we can also have a look at Activity Runs info.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhodApJ1xUsg5KKLaLLCC8t3CjaStUg5sA8sX3cGoPSfEvjcbLSk4oOLwk1DPTRCerdEqv_a9f8S43EczOeA-LeJugHxLa0v6LwcY953o8vf8Byoh7K1oyuufwiz2uWLKaDJCsP9cJn8TUl/s1600/4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="339" data-original-width="1330" height="162" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhodApJ1xUsg5KKLaLLCC8t3CjaStUg5sA8sX3cGoPSfEvjcbLSk4oOLwk1DPTRCerdEqv_a9f8S43EczOeA-LeJugHxLa0v6LwcY953o8vf8Byoh7K1oyuufwiz2uWLKaDJCsP9cJn8TUl/s640/4.JPG" width="640" /></a></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Another thing is if we click on {} code icon on the UI, we can get the corresponding JSON document of pipeline / dataset.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">Initially we were able to </span><a href="http://www.allaboutmssql.com/2017/12/lift-and-shift-ssis-packages-to-azure.html" style="font-family: Arial, Helvetica, sans-serif;" target="_blank"><i>provision the Azure-SSIS integration runtime using powershell commands</i></a><span style="font-family: "arial" , "helvetica" , sans-serif;">, now we have an </span><a href="https://docs.microsoft.com/en-us/azure/data-factory/tutorial-create-azure-ssis-runtime-portal" style="font-family: Arial, Helvetica, sans-serif;" target="_blank"><i>UI</i></a><span style="font-family: "arial" , "helvetica" , sans-serif;"> to do that.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><i><br /></i></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><a href="https://docs.microsoft.com/en-us/azure/data-factory/control-flow-system-variables" target="_blank"><i>System variables supported by Azure Data Factory</i></a></span><br />
<h3 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: small; font-weight: normal;">Reference: </span></h3>
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="https://azure.microsoft.com/en-us/blog/adf-v2-visual-tools-enabled-in-public-preview/" target="_blank">Azure Data Factory: Visual Tools enabled in public preview</a></i></span></li>
</ul>
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: small; font-weight: normal;">See Also: </span></h3>
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif;"><i><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-51654162688492781532018-01-14T23:48:00.001+05:302018-01-14T23:48:29.112+05:30How to concatenate values with specified delimiter<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: Arial, Helvetica, sans-serif;">In SQL Server 2017, new function is introduced to concatenate values with specified delimiter</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In <a href="http://www.allaboutmssql.com/2012/10/sql-server-2012-concat-string-function.html" target="_blank"><i>SQL Server 2012, Concat()</i></a> was introduced. Only difference between Concat() and Concat_WS(), </span><span style="font-family: Arial, Helvetica, sans-serif;">Concat_WS() allows delimiter to be specified to concatenate values.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Both CONCAT & CONCAT_WS ignores the <i><a href="http://www.allaboutmssql.com/2013/09/sql-server-concatenating-string-integer.html" target="_blank">SET CONCAT_NULL_YIELDS_NULL {ON|OFF} setting</a> </i>as shown in below image.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"></span><br />
<a name='more'></a><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh96km8BgaEb-_BZV9ztKu5reI_u3IrlOBFLsPewLCz0-rzQNV9q_dzxXISzAUJfyIl__WO4QYIj5U0Cs4tQfRpIfY99ZymNvFEttHqynuVJg5Hq-bjK62sPiG42Xc8CfDyxjOy7U8GdtTc/s1600/sp4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="598" data-original-width="316" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh96km8BgaEb-_BZV9ztKu5reI_u3IrlOBFLsPewLCz0-rzQNV9q_dzxXISzAUJfyIl__WO4QYIj5U0Cs4tQfRpIfY99ZymNvFEttHqynuVJg5Hq-bjK62sPiG42Xc8CfDyxjOy7U8GdtTc/s640/sp4.JPG" width="338" /></a></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Concat_WS() requires atleast three arguments:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglNX1_LjukAqDruONRt3_CcA8N19I4fR6iUW5N4pObtYHqzxFMSEQuplAwC2fiYfm1OVF5Cxf0_4CVJb3gOWnIjjWAOWMPVeAE_6t12r8oyjQF04pUj8KWa3_86kAcFsFoWRE1oip9XYwU/s1600/sp5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="364" data-original-width="412" height="564" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglNX1_LjukAqDruONRt3_CcA8N19I4fR6iUW5N4pObtYHqzxFMSEQuplAwC2fiYfm1OVF5Cxf0_4CVJb3gOWnIjjWAOWMPVeAE_6t12r8oyjQF04pUj8KWa3_86kAcFsFoWRE1oip9XYwU/s640/sp5.JPG" width="640" /></a></div>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<ul style="text-align: left;">
<li><a href="https://docs.microsoft.com/en-us/sql/t-sql/functions/concat-transact-sql" target="_blank"><i><span style="font-family: Arial, Helvetica, sans-serif;">CONCAT()</span></i></a></li>
<li><a href="https://docs.microsoft.com/en-us/sql/t-sql/functions/concat-ws-transact-sql" target="_blank"><i><span style="font-family: Arial, Helvetica, sans-serif;">CONCAT_WS()</span></i></a></li>
</ul>
<br />
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">See Also:</span></h3>
<br />
<ul style="text-align: left;">
<li><i><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/p/t-sql-scripts_19.html" target="_blank">My Posts on T-SQL</a></span></i></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-46637002198380123152018-01-14T22:39:00.001+05:302018-01-14T22:39:33.357+05:30How to combine separate row values into delimited string<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In one of my old post, I wrote about <a href="http://www.allaboutmssql.com/2012/10/sql-server-combine-separate-row-values.html" target="_blank"><i>how to combine separate row values into delimited string</i></a>. Now starting from SQL Server 2017, new string function was introduced to perform this string aggregation easily. Let us see few examples in this post.</span><br />
<br />
<pre class="brush:sql">DECLARE @Tmp TABLE (ID INT,
Name VARCHAR(30))
INSERT INTO @Tmp SELECT 1,'SQL Server'
INSERT INTO @Tmp SELECT 2,'Oracle'
INSERT INTO @Tmp SELECT 3,'DB2'
SELECT * FROM @Tmp
SELECT STRING_AGG (Name, ',') FROM @Tmp
</pre>
<br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;">We can also order the concatenated values as shown in below example:</span><br />
<br />
<pre class="brush:sql">--To order the concatenated values
SELECT STRING_AGG (Name, ',') WITHIN GROUP (ORDER BY Name ASC) AS DBNames FROM @Tmp
</pre>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfpYQAZ0L0-2E2Ny1oCnwqRiPUnyFa7He8j79JlIONC14320iaQ_grbq4CQ-iYrNy439C19TAc3PgP3SBRdNGlyiemx5zgRfpQjizTQRIQ7NxUhYFQz3mYx_KljrU0VPw8281K3eF-MH-b/s1600/sp2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="603" data-original-width="640" height="602" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfpYQAZ0L0-2E2Ny1oCnwqRiPUnyFa7He8j79JlIONC14320iaQ_grbq4CQ-iYrNy439C19TAc3PgP3SBRdNGlyiemx5zgRfpQjizTQRIQ7NxUhYFQz3mYx_KljrU0VPw8281K3eF-MH-b/s640/sp2.JPG" width="640" /></a></div>
<br />
<h4 style="text-align: left;">
<span style="font-family: "arial" , "helvetica" , sans-serif;">Example on how to combine values from separate grouped records into delimited string.</span></h4>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<br />
<pre class="brush:sql">DECLARE @Tmp TABLE (ID INT,
DBName VARCHAR(30))
INSERT INTO @Tmp SELECT 1,'SQL Server 2005'
INSERT INTO @Tmp SELECT 1,'SQL Server 2008'
INSERT INTO @Tmp SELECT 1,'SQL Server 2008R2'
INSERT INTO @Tmp SELECT 1,'SQL Server 2012'
INSERT INTO @Tmp SELECT 2,'Oracle 10g'
INSERT INTO @Tmp SELECT 2,'Oracle 11g'
INSERT INTO @Tmp SELECT 2,'Oracle 12c'
INSERT INTO @Tmp SELECT 3,'DB2 v9.7'
INSERT INTO @Tmp SELECT 3,'DB2 v9.8'
SELECT ID, STRING_AGG (DBName, '|') DBName FROM @Tmp
GROUP BY ID
</pre>
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAGbJmk55_0qi6CHHMGsBrsDZTg8eke5eBcwv_nimSjhCzIZ0QZPRimD0IvYhpISEkDupRhJCjNpu9PE8jSYlceQgW8KCiZiOWw09foFZt3prTr_hZi4kbJc7tra3iNfHjYjBDSkQlRNjs/s1600/sp3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="615" height="622" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgAGbJmk55_0qi6CHHMGsBrsDZTg8eke5eBcwv_nimSjhCzIZ0QZPRimD0IvYhpISEkDupRhJCjNpu9PE8jSYlceQgW8KCiZiOWw09foFZt3prTr_hZi4kbJc7tra3iNfHjYjBDSkQlRNjs/s640/sp3.JPG" width="640" /></a></div>
<br />
<br />
<h3 style="text-align: left;">
Reference: </h3>
<br />
<ul style="text-align: left;">
<li><i><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://docs.microsoft.com/en-us/sql/t-sql/functions/string-agg-transact-sql" target="_blank">STRING_AGG </a></span></i></li>
</ul>
<br />
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">See Also: </span></h3>
<br />
<ul style="text-align: left;">
<li><i><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/p/t-sql-scripts_19.html" target="_blank">My posts on T-SQL</a></span></i></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-68727461471580381802018-01-14T19:43:00.001+05:302018-01-14T19:43:34.243+05:30How to split delimited string into separate row<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: "arial" , "helvetica" , sans-serif;">In this post, let us see how to split delimited string into separate row. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">In previous versions of SQL Server, this can be achieved using <i><a href="http://www.allaboutmssql.com/2012/10/sql-server-split-csv-into-separate-row.html" target="_blank">User defined function</a>. </i></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">When support for querying JSON data was introduced in SQL Server, using <i><a href="http://www.allaboutmssql.com/2015/12/sql-server-split-csv-into-separate-row.html" target="_blank">OPENJSON</a> </i>seems to be simpler technique to split delimited string into separate row.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">Finally starting from SQL Server 2016, new built string function was introduced to do this. Let us see few examples with various delimiters:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<a name='more'></a><span style="font-family: "arial" , "helvetica" , sans-serif;"><br />
</span><br />
<br />
<pre class="brush:sql">SELECT * FROM STRING_SPLIT('SQL Server, Oracle, DB2',',')
SELECT * FROM STRING_SPLIT('SQLServer Oracle DB2',' ')
SELECT * FROM STRING_SPLIT('SQL Server|Oracle|','|')
WHERE value <> ''
</pre>
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo0h8njLroUH2TBuKq7XtTXVTvoPNwoXduzVIXK1NHQfLc4IJ93VKZzAiIY6zTks7RoEyu7bYkrsY7YYPmTzn7auyI6Fupdx9wIDO5FEZTGwEoLdlfW3biYUukd5K7KVdvFs_6u__Qf-ct/s1600/sp1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="699" data-original-width="833" height="536" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo0h8njLroUH2TBuKq7XtTXVTvoPNwoXduzVIXK1NHQfLc4IJ93VKZzAiIY6zTks7RoEyu7bYkrsY7YYPmTzn7auyI6Fupdx9wIDO5FEZTGwEoLdlfW3biYUukd5K7KVdvFs_6u__Qf-ct/s640/sp1.JPG" width="640" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">This function will be very handy in SSRS scenarios where SSRS dataset gets delimited string as input from multi valued parameter.</span><br />
<br />
<ul style="text-align: left;">
<li><span style="font-family: Arial, Helvetica, sans-serif;"><i><a href="http://www.allaboutmssql.com/2013/09/ssrs-multi-valued-parameter-as-stored.html" target="_blank">SSRS - Multi-valued Parameter as Stored procedure Input</a></i></span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;"><i><a href="http://www.allaboutmssql.com/2013/05/passing-comma-separated-values-ssrs.html" target="_blank">Passing comma separated values (SSRS - multivalued parameter) as input to the stored procedure</a> </i></span></li>
</ul>
<br />
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: small;">Reference: </span></h3>
<br />
<ul style="text-align: left;">
<li><i><span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://docs.microsoft.com/en-us/sql/t-sql/functions/string-split-transact-sql" target="_blank">STRING_SPLIT</a></span></i></li>
</ul>
<br />
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: small;">See Also: </span></h3>
<br />
<ul style="text-align: left;">
<li><span style="font-family: Arial, Helvetica, sans-serif;"><i><a href="http://www.allaboutmssql.com/p/t-sql-scripts_19.html" target="_blank">My posts on T-SQL</a></i></span></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0tag:blogger.com,1999:blog-954987329016201413.post-54817566800770890202018-01-14T03:04:00.001+05:302018-01-14T03:04:03.533+05:30How to load data into Azure SQL Data Warehouse using SSIS<div dir="ltr" style="text-align: left;" trbidi="on">
<span style="font-family: Arial, Helvetica, sans-serif;">In this post, let us see how to load files from local folder into Azure SQL Data Warehouse using <b>Azure SQL DW Upload Task</b> in SSIS.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">We need to download and install <a href="http://www.allaboutmssql.com/2018/01/how-to-upload-files-into-azure-blob.html" target="_blank"><i>Azure Feature pack to get the Azure components in SSIS</i></a>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><b><br /></b></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><b>Azure SQL DW Upload Task</b> is designed to work in such a way that it consumes the files in local folder and places in Azure Blob storage as staging (there is an option to retain files within Blob storage)</span><span style="font-family: Arial, Helvetica, sans-serif;"> and then it is loaded into Azure SQL data warehouse.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"></span><br />
<a name='more'></a><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">After creating an SSIS package, drag & drop </span><b style="font-family: Arial, Helvetica, sans-serif;">Azure SQL DW Upload Task </b><span style="font-family: Arial, Helvetica, sans-serif;">into Control Flow.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In the task editor, first let us provide source information - Local Directory, Recursively, FileName</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjxqPlTDI5i6nII4TIuxltQ3QoFO3fqVfefr-og5ZW1yOhQBOw00TolgepduShEbrFnCGOmHu2Eo_-qwDDS-joyGmXia8Ng3szW2dBeN96CH14lRQKZPU_jVI9Gyu39ro6k2xyq6Q-ul9q/s1600/ADW4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="257" data-original-width="740" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjxqPlTDI5i6nII4TIuxltQ3QoFO3fqVfefr-og5ZW1yOhQBOw00TolgepduShEbrFnCGOmHu2Eo_-qwDDS-joyGmXia8Ng3szW2dBeN96CH14lRQKZPU_jVI9Gyu39ro6k2xyq6Q-ul9q/s640/ADW4.JPG" width="640" /></a><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Create connection to Azure Blob storage and provide Blob storage details:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGIM3a3kqcmPO6zwD4EMrbDPZacUIAAPkLU19tUEqAa_Vu-S86ytIWFfQ-_RPR24N95EqaYWEeie6n8cozoauQaQH0ruZ7ZQw2x9pR4OnuilqIgGbRNHIRd14OB3SClcNXoLuRX2obbshs/s1600/ADW1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="664" data-original-width="1253" height="338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGIM3a3kqcmPO6zwD4EMrbDPZacUIAAPkLU19tUEqAa_Vu-S86ytIWFfQ-_RPR24N95EqaYWEeie6n8cozoauQaQH0ruZ7ZQw2x9pR4OnuilqIgGbRNHIRd14OB3SClcNXoLuRX2obbshs/s640/ADW1.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Create connection to Azure SQL data warehouse</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgboenNlkcq5hy2hnV0m7USvU59hT3wc-_fGVi-DvXHIhitZTGW6S67PqyLVt9d7UvW-JciQpQAbgAl6NINZed1v6qUe8_Y3kK_4BjxIjTKl7wZMBc48MYkvzkYBhFb4Qb2w0o399YtchHW/s1600/ADW2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="680" data-original-width="736" height="590" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgboenNlkcq5hy2hnV0m7USvU59hT3wc-_fGVi-DvXHIhitZTGW6S67PqyLVt9d7UvW-JciQpQAbgAl6NINZed1v6qUe8_Y3kK_4BjxIjTKl7wZMBc48MYkvzkYBhFb4Qb2w0o399YtchHW/s640/ADW2.JPG" width="640" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">We also have an option to create destination table on the fly (while package is executed) by mentioning the table name as shown below:</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM4VhybPZUdW5kDTw4SMbWj5w694d4MYRcl5ROHgj2wkCxjkW2pqZrzR6rZxgzYPVhtG9b3Xyf90xQJCXDl87rrnwGGK5haf-cx5bXiBm-IlAMRsu_i4cS6KxiGI7_LdWKP9ZZqzwWihe5/s1600/ADW3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="300" data-original-width="728" height="262" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM4VhybPZUdW5kDTw4SMbWj5w694d4MYRcl5ROHgj2wkCxjkW2pqZrzR6rZxgzYPVhtG9b3Xyf90xQJCXDl87rrnwGGK5haf-cx5bXiBm-IlAMRsu_i4cS6KxiGI7_LdWKP9ZZqzwWihe5/s640/ADW3.JPG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">In the task editor, T-SQL shows the script to load the data from Azure blob to Azure SQL data warehouse. We can also edit the script as shown below:</span></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkrbVCQ9SSODbhwwFvYF9NB9jAmKm2FhknLTssAYYG3niedbO6bKiJm2womVdba8ADzzv6Q6ouYiL0J_Pt3KuZG1n4k1l9hTPIEVyI6wEKDCflxSl8Fp2pIg2RCMhdbbs6vx70oIUWsg-a/s1600/ADW5.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="628" data-original-width="909" height="442" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkrbVCQ9SSODbhwwFvYF9NB9jAmKm2FhknLTssAYYG3niedbO6bKiJm2womVdba8ADzzv6Q6ouYiL0J_Pt3KuZG1n4k1l9hTPIEVyI6wEKDCflxSl8Fp2pIg2RCMhdbbs6vx70oIUWsg-a/s640/ADW5.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">By looking at above script, we can notice that <i><a href="http://www.allaboutmssql.com/2017/12/how-to-load-data-from-azure-data-lake.html" target="_blank">Create table as Select</a> </i>mechanism is used to load the data into Azure SQL DW.</span><br />
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">When I execute the package, it failed with below error message:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT4dNNfaqXTTV5up0zBTXap6p6pHOaagCPlwfZ1zyB_hdZ_QpowuGGu8N7fYW3A-CZ03rs11ww4qEldVUViswPTY0BH291o5Yvo5FNCiztLiOsdmdDkv5vj5ngw9TOuepkR3jGM9fpF1Fl/s1600/ADW6.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="382" data-original-width="1101" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiT4dNNfaqXTTV5up0zBTXap6p6pHOaagCPlwfZ1zyB_hdZ_QpowuGGu8N7fYW3A-CZ03rs11ww4qEldVUViswPTY0BH291o5Yvo5FNCiztLiOsdmdDkv5vj5ngw9TOuepkR3jGM9fpF1Fl/s640/ADW6.JPG" width="640" /></a></div>
<br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">Created master key to resolve above error:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcQfEib8C6dgyK16JDb3jIXUYUkmll22zp9ncOrdUi4iJysuyJCBIhCi6DfU6vhcywG7uHMGLoG2x2GO35Yj4t05Re4bxR_JmtQCxpf8lDMTQfjAJnZreeFaYr870lPcjMmt-mtXDnfX3Q/s1600/ADW7.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="399" data-original-width="697" height="366" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcQfEib8C6dgyK16JDb3jIXUYUkmll22zp9ncOrdUi4iJysuyJCBIhCi6DfU6vhcywG7uHMGLoG2x2GO35Yj4t05Re4bxR_JmtQCxpf8lDMTQfjAJnZreeFaYr870lPcjMmt-mtXDnfX3Q/s640/ADW7.JPG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUeNHN9gydf9fmG1TeL0Vn7hI2hJvRS4gYitSzJINLetCkmGQ3KMueCdylLeKMUXyG1c2mbHTaLi8W6FtYqz-3zVT2O3pVCgflZGAx7Of3h06No09nrMyOsngC6ZuP24zUpcSWDlp9rHN_/s1600/ADW8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="650" data-original-width="650" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUeNHN9gydf9fmG1TeL0Vn7hI2hJvRS4gYitSzJINLetCkmGQ3KMueCdylLeKMUXyG1c2mbHTaLi8W6FtYqz-3zVT2O3pVCgflZGAx7Of3h06No09nrMyOsngC6ZuP24zUpcSWDlp9rHN_/s640/ADW8.JPG" width="640" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">As shown in below image, after successful execution of package, from SSMS we can see Table, External Data Source, External File Format are created.</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEix1jW44lLxmRCY-I2zIrzji0ACU00Z12YpXFQxrkvwKLqHECikyZvTtk9FuQiFCrfyYdqXBvxhAZp0cCsbZQNwwfaOuf8s31V1YdQq2jvqGilSAoOnJu7-APOFhqPAI71vDn9GMvd0fKVI/s1600/ADW9.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="590" data-original-width="899" height="420" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEix1jW44lLxmRCY-I2zIrzji0ACU00Z12YpXFQxrkvwKLqHECikyZvTtk9FuQiFCrfyYdqXBvxhAZp0cCsbZQNwwfaOuf8s31V1YdQq2jvqGilSAoOnJu7-APOFhqPAI71vDn9GMvd0fKVI/s640/ADW9.JPG" width="640" /></a></div>
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">All setting of Azure SQL DW Task:</span><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaGGP9Z_6Nb11VxORQyCJcZH7NQ30NSGpZ5JS2AKyVlFTbjLvyyN0Im5VO93Eon8lTkdGV2-0G-FW4V75FYmbUqLkHbzTeMvzmpaUD4RlTH__aZsbYQlqnWpxtPRdUO5ARfBQZGJnAHBPK/s1600/ADW10.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="727" data-original-width="1366" height="340" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaGGP9Z_6Nb11VxORQyCJcZH7NQ30NSGpZ5JS2AKyVlFTbjLvyyN0Im5VO93Eon8lTkdGV2-0G-FW4V75FYmbUqLkHbzTeMvzmpaUD4RlTH__aZsbYQlqnWpxtPRdUO5ARfBQZGJnAHBPK/s640/ADW10.JPG" width="640" /></a></div>
<br />
<h3 style="text-align: left;">
<span style="font-family: Arial, Helvetica, sans-serif;">See Also: </span></h3>
<br />
<ul style="text-align: left;">
<li><i><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/p/c.html" target="_blank">My posts on Microsoft Azure</a></span></i></li>
<li><i><span style="font-family: Arial, Helvetica, sans-serif;"><a href="http://www.allaboutmssql.com/p/sql-server-integration-services.html" target="_blank">My posts on SSIS</a></span></i></li>
</ul>
</div>
Sathyahttp://www.blogger.com/profile/14340039326216838476noreply@blogger.com0