Showing posts with label rows. Show all posts
Showing posts with label rows. Show all posts

Thursday, March 29, 2012

Comparing two sets of data

I have the following situation. One set of data has 274 rows (set2)
and anther has 264 (set1). Both data sets are similar in structure as
well as values for both of them were extracts from the same parent
table. Hope the info would substitute DDL. I need to find the "gap"
rows between these two sets.
Attempted to run a query like
select count(*)
from set2
where not exists
(select *
from set1)
did not yield what I desired. What else to try?

TIA.NickName wrote:
> I have the following situation. One set of data has 274 rows (set2)
> and anther has 264 (set1). Both data sets are similar in structure as
> well as values for both of them were extracts from the same parent
> table.
> Hope the info would substitute DDL.

It doesn't.

> I need to find the "gap"
> rows between these two sets.
> Attempted to run a query like
> select count(*)
> from set2
> where not exists
> (select *
> from set1)
> did not yield what I desired. What else to try?

Try posting your DDL. We at least need column names and keys to help you
here.

Zach|||OK,

I've proven that the "EXISTS" keyword/function can't solve this
problem. But then what?

-- test equality between two data sets
-- desired resultset: return "gap" rows

-- DDL and DML
create table #tmp1 (col1 int, col2 char(1));

insert into #tmp1
values(1,'a')
insert into #tmp1
values(2,'b')
insert into #tmp1
values(3,'c')
insert into #tmp1
values(4,'d')
insert into #tmp1
values(5,'e');

select * into #tmp2
from #tmp1
where col1 < 5;
select *
from #tmp1
where not exists
(select *
from #tmp2)|||NickName (dadada@.rock.com) writes:
> I've proven that the "EXISTS" keyword/function can't solve this
> problem. But then what?

It certainly can, but you must specify how the NOT EXISTS is to work.
SQL is not about telepathy.

For your repro, you could do

select *
from #tmp1 t1
where not exists (Select *
from #tmp2 t2
WHERE t1.col1 = t2.col1)

or

select *
from #tmp1 t1
where not exists (Select *
from #tmp2 t
WHERE t1.col1 = t2.col1
AND t1.col2 = t2.col2)

depending on what you are looking for. The first just lists missing key
values, the second list all mismatches.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||Thanks, Erland.
*) laziness is an enemy, I thought about that but the table has a large
number of columns, so, was wondering if there's another approach. Will
try it tomorrow when the db is available.
*) btw, I like the word, "telepathy".

Erland Sommarskog wrote:
> NickName (dadada@.rock.com) writes:
> > I've proven that the "EXISTS" keyword/function can't solve this
> > problem. But then what?
> It certainly can, but you must specify how the NOT EXISTS is to work.
> SQL is not about telepathy.
> For your repro, you could do
> select *
> from #tmp1 t1
> where not exists (Select *
> from #tmp2 t2
> WHERE t1.col1 = t2.col1)
> or
> select *
> from #tmp1 t1
> where not exists (Select *
> from #tmp2 t
> WHERE t1.col1 = t2.col1
> AND t1.col2 = t2.col2)
> depending on what you are looking for. The first just lists missing
key
> values, the second list all mismatches.
>
> --
> Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
> Books Online for SQL Server SP3 at
> http://www.microsoft.com/sql/techin.../2000/books.asp|||NickName (dadada@.rock.com) writes:
> Thanks, Erland.
> *) laziness is an enemy, I thought about that but the table has a large
> number of columns, so, was wondering if there's another approach. Will
> try it tomorrow when the db is available.

If your aim is to find differences in any column, you will indeed have
to write code that has all column. There is no shortcut. What you can
do, if you have many tables and columns, is to generate code by reading
metadata. But this requires that you have a clear understanding for which
columns you want to compare, and which you do not.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||Erland,

Something seems odd. Here's two test cases. Both are successful.
-- test equality between two data sets (with exact meta data)
-- desired resultset: return "gap" rows

-- DDL and DML
create table #tmp1 (col1 int, col2 char(1));

insert into #tmp1
values(1,'a')
insert into #tmp1
values(2,'b')
insert into #tmp1
values(3,'c')
insert into #tmp1
values(4,'d')
insert into #tmp1
values(5,'e');

select * into #tmp2
from #tmp1
where col1 < 5;

select *
from #tmp1 t1
where not exists
(select *
from #tmp2 t2
where t1.col1 = t2.col1
and t1.col2 = t2.col2)

-- outcome = success

-- test equality between two data sets
-- with minor meta data difference (one table has 3 attributes while
the other has 2)
-- note: comparison seems still successful
-- desired resultset: return "gap" rows

drop table #tmp1
drop table #tmp2;

-- DDL and DML
create table #tmp1 (col1 int, col2 char(1),col3 int);

insert into #tmp1
values(1,'a',11)
insert into #tmp1
values(2,'b',22)
insert into #tmp1
values(3,'c',33)
insert into #tmp1
values(4,'d',44)
insert into #tmp1
values(5,'e',55);

select col1,col2 into #tmp2
from #tmp1
where col1 < 5;

select *
from #tmp1 t1
where not exists
(select *
from #tmp2 t2
where t1.col1 = t2.col1
and t1.col2 = t2.col2)

-- outcome = success

HOWEVER, when I applied the above to my tables (sorry I can't post
exact structure nor data here, PARENT table has 13 columns and the
derived table has 12 of them, comparison is between them), sql still
failed to find the "gap". What could possibly stands in the way?
Many thanks.|||Ahe, I think I've found the problem. In "my" tables, both has
duplicate rows, however, the number of duplicate rows are not the same,
(distinct rows are sure the same).|||NickName (dadada@.rock.com) writes:
> Ahe, I think I've found the problem. In "my" tables, both has
> duplicate rows, however, the number of duplicate rows are not the same,
> (distinct rows are sure the same).

I'm not sure how this should be addressed. Are you saying that in
one table you have two rows with the same value, but in another you
only have one?

If there are no disctinct keys in the data, all relational operations will
be problematic, that's for sure.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||You might find it easier to export the table contents as ordered data and
then use a text file compare utility. Command-prompt example:

BCP "SELECT * FROM MyDatabase..Table1 ORDER BY MyPK" queryout
"C:\temp\Table1.txt" /T /c /SMyServer

BCP "SELECT * FROM MyDatabase..Table2 ORDER BY MyPK" queryout
"C:\temp\Table2.txt" /T /c /SMyServer

WINDIFF "C:\temp\Table1.txt" "C:\temp\Table2.txt"

--
Happy Holidays

Dan Guzman
SQL Server MVP

"NickName" <dadada@.rock.com> wrote in message
news:1104161334.766126.171470@.f14g2000cwb.googlegr oups.com...
> Ahe, I think I've found the problem. In "my" tables, both has
> duplicate rows, however, the number of duplicate rows are not the same,
> (distinct rows are sure the same).|||Dan Guzman (guzmanda@.nospam-online.sbcglobal.net) writes:
> You might find it easier to export the table contents as ordered data and
> then use a text file compare utility. Command-prompt example:
> BCP "SELECT * FROM MyDatabase..Table1 ORDER BY MyPK" queryout
> "C:\temp\Table1.txt" /T /c /SMyServer
> BCP "SELECT * FROM MyDatabase..Table2 ORDER BY MyPK" queryout
> "C:\temp\Table2.txt" /T /c /SMyServer
> WINDIFF "C:\temp\Table1.txt" "C:\temp\Table2.txt"

However, the use of windiff will break down if there is a column which is
permitted to be different in the tables. But there is a better alternative:
Beyond Compare, from http://www.scootersoftware.com. Beyond Compare
offers comparison on character level.

--
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techin.../2000/books.asp|||Erland,

You're exactly right. Maybe they did that to test my analytic skill,
who knows :)|||Dan,

Thanks. It seems a good way to try. Will do tomorrow.

Erland, no columns are supposed to contain different data sets though
one table may have an extra column, that is supposedly the only
difference. OK, I'll try the recommended software as well when get a
chance. I appreciate it.

Donsqlsql

Comparing two rows in a Table

Hi Gurus,
Is there any automated way to compare the contents of TWO rows in the same
table of SQL Server ?
Thanks in advance.
Regards,
YogWhat do mean by "Automated"? There is certainly a way, just write a query
that returns a boolean 1/0 based on whether the row column values are the
same or different... How to "Automate" it, depends on what "event" you want
to "trigger", (cause ) it to run... If yo want it to run automatically on
some time interval, use SQLAgent...
If you want it to run whenever someone inserts, updates, or deletes a record
from the table, then use a trugger.
"Yog" wrote:

> Hi Gurus,
> Is there any automated way to compare the contents of TWO rows in the same
> table of SQL Server ?
> Thanks in advance.
> Regards,
> Yog|||What result do you want from the comparison? Do you mean you want to
compart two particular rows? In that case you can use a self-join on
the columns you want to compare.
Maybe you just want to find duplicates, in which case:
SELECT col1, col2, col3, ...
FROM YourTable
GROUP BY col1, col2, col3, ...
HAVING COUNT(*)>1
David Portas
SQL Server MVP
--|||Yog wrote:
> Hi Gurus,
> Is there any automated way to compare the contents of TWO rows in the
> same table of SQL Server ?
> Thanks in advance.
> Regards,
> Yog
You can use the CHECKSUM() function as in:
Create table #checktest(col1 int, col2 int, col3 int)
insert into #checktest values (1, 2, 2)
insert into #checktest values (2, 2, 2)
insert into #checktest values (3, 2, 3)
Select CHECKSUM(col2, col3) as "ID 1"
From #checktest
where col1 = 1
UNION ALL
Select CHECKSUM(col2, col3) as "ID 2"
From #checktest
where col1 = 2
UNION ALL
Select CHECKSUM(col2, col3) as "ID 3"
From #checktest
where col1 = 3
drop table #checktest
David Gugick
Imceda Software
www.imceda.com|||To precisely put it , the content of the problem is as below
Objective :- Compare TWO rows contents for few selected Columns. If they are
different then show those columns with the contents.
ENV:- SQL Server 2000 , T-SQL, SQL Server Tools (No other programming
language or front end)
"CBretana" wrote:
> What do mean by "Automated"? There is certainly a way, just write a quer
y
> that returns a boolean 1/0 based on whether the row column values are the
> same or different... How to "Automate" it, depends on what "event" you wa
nt
> to "trigger", (cause ) it to run... If yo want it to run automatically on
> some time interval, use SQLAgent...
> If you want it to run whenever someone inserts, updates, or deletes a reco
rd
> from the table, then use a trugger.
> "Yog" wrote:
>|||SELECT A.col1, A.col2, ...
FROM YourTable AS A,
YourTable AS B
WHERE A.x = ?
AND B.x = ? /* Specify the two rows to be compared */
AND
(A.col1<>B.col1
OR A.col1<>B.col1
OR ...)
David Portas
SQL Server MVP
--|||Hi David,
Many thanks for your POST.
Your answer provided me HINT to achieve my goal of comparing two rows for my
requirement.
Regards,
Yog
"David Portas" wrote:

> SELECT A.col1, A.col2, ...
> FROM YourTable AS A,
> YourTable AS B
> WHERE A.x = ?
> AND B.x = ? /* Specify the two rows to be compared */
> AND
> (A.col1<>B.col1
> OR A.col1<>B.col1
> OR ...)
> --
> David Portas
> SQL Server MVP
> --
>

Comparing two rows in a Table

Hi Gurus,
Is there any automated way to compare the contents of TWO rows in the same
table of SQL Server ?
Thanks in advance.
Regards,
Yog
On Thu, 17 Mar 2005 10:19:02 -0800, Yog wrote:

>Hi Gurus,
>Is there any automated way to compare the contents of TWO rows in the same
>table of SQL Server ?
Hi Yog,
The answer is probably yes - but you've given too little information
about your needs to allow for any more specific information.
Please check out www.aspfaq.com/5006to find out what information is
needed (and in what form) to allow others to understand your problem and
proposes solutions.
Best, Hugo
(Remove _NO_ and _SPAM_ to get my e-mail address)
|||Well,
To precisely put it , the content of the problem is as below
Objective :- Compare TWO rows contents for few selected Columns. If they are
different then show those columns with the contents.
ENV:- SQL Server 2000 , T-SQL, SQL Server Tools (No other programming
language or front end)
Hope this makes sence.
"Hugo Kornelis" wrote:

> On Thu, 17 Mar 2005 10:19:02 -0800, Yog wrote:
>
> Hi Yog,
> The answer is probably yes - but you've given too little information
> about your needs to allow for any more specific information.
> Please check out www.aspfaq.com/5006to find out what information is
> needed (and in what form) to allow others to understand your problem and
> proposes solutions.
> Best, Hugo
> --
> (Remove _NO_ and _SPAM_ to get my e-mail address)
>
|||On Thu, 17 Mar 2005 22:27:02 -0800, Yog wrote:

>Well,
>To precisely put it , the content of the problem is as below
>Objective :- Compare TWO rows contents for few selected Columns. If they are
>different then show those columns with the contents.
>ENV:- SQL Server 2000 , T-SQL, SQL Server Tools (No other programming
>language or front end)
>Hope this makes sence.
Hi Yog,
Yes - but not enough to provide actual help. I need your table structure
(posted as CREATE TABLE stmt), some sample data (as INSERT stmts) and
the expected output.
I see that the link I posted got mangled - I missed a space between the
link and the next word. Here's the link again:
www.aspfaq.com/5006
Please follow that link and read the information there; then post the
required information here for further help.
Best, Hugo
(Remove _NO_ and _SPAM_ to get my e-mail address)

comparing two fields

Greetings.

I am trying to write a sql that identifies all the rows in one table that do not match another. The sql needs to compare the values in two columns of each table. In the past, I've used a where not exists and a subquery to identify missing rows.

select a.block_code

from BLKPTS_ADDXY a

where not exists

(select block_code

from allblkzips as s

where s.block_code = a.block_code)

I need to add a second column from each table (zip) to the query. For some reason, it doesn't seem as straightforward as I thought it should be. Suggestions are appreciated.

Thanks.

alan

If you are using SQL 2005, try a EXCEPT JOIN. Something like:

SELECT Block_Code, Zip
FROM BLKPTS_ADDXY
EXCEPT
SELECT Block_Code, Zip
FROM allblkzips

|||I usually use a not in clause

I m pretty sure this will work..you can concat the two columns thus,

select convert(varchar(10),a.block_code)+convert(varchar(10),a.second_code), <anything else you want to select>

from BLKPTS_ADDXY a

where convert(varchar(10),a.block_code)+convert(varchar(10),a.second_code) not in

(

selectconvert(varchar(10),s.block_code)+convert(varchar(10),s.second_code)

from allblkzips as s

)

This will probably run faster than the query you are using which tries to run a where condition in an inner query with a field in the outer query, comparable with a for followed by another v/s a nested for loop in procedural programming.|||

Do either of the queries below return the results you are expecting?

Chris

SELECT a.block_code

FROM BLKPTS_ADDXY a

WHERE NOT EXISTS

(SELECT block_code

FROM allblkzips s

WHERE s.block_code = a.block_code

AND s.zip = a.zip)

SELECT a.block_code

FROM BLKPTS_ADDXY a

WHERE NOT EXISTS

(SELECT block_code

FROM allblkzips s

WHERE s.block_code = a.block_code)

OR NOT EXISTS

(SELECT block_code

FROM allblkzips s

WHERE s.zip = a.zip)

|||

I prefer the "outer join" method, since it helps enforce thinking in terms of INNER and OUTER JOINs.

select

a.block_code

from BLKPTS_ADDXY a

left outer join allblkzips s

on a.block_code = s.block_code and

a.zip = s.zip

where (s.zip is null) -- you only need to specify one column here, since the JOIN fails if both do not match

Dan

Comparing to Resultsets

I have a situation where I have two tables, one in Access and one in SQL
Server. There are several hundred thousand rows and the tables should
contain the exact same number of rows. They may not though and the count
could be off by one row or a hundred rows. I need to be able to identify
which rows are not in both resultsets so i can update the second table with
the missing rows.
My problem is that I have built a VB app that SELECT * FROM table and then I
query the second table using the current results from table 1... Select *
From table where companyName = 'value from table1'
This works but it has taken 5 hours so far and isn't even half way done.
The IDs in both table can be (and are) different in both tables so i don't
really have a solid way to compare records on a record by record basis. To
make matters worse Access and SQL server appear to have different sort
engines so when I sort the records by a column, the fields are not in
exactly the same order.
What options do i have to compare these two tables and identify the rows
that are different'
Thanks,
RonYou'd have to post DDL to help us help you.
You don't really need a single common column to compare result sets, they
must, however, contain comparable data.
ML
http://milambda.blogspot.com/

Tuesday, March 27, 2012

Comparing Smalldatetime column

I have two Smalldatetime columns. One of the columns have
NULL values in some of the rows. When I compare the two
columns, I don't get the rows that have NULL values (I am
comparing them to be NOT the same e.g Select * from
mytable where column1 <> column2). I get all the rows that
are not the same but have values but not the NULL columns.
How can I get the NULL columns too '
Thanks for help.IS NULL and IS NOT NULL are the predicates used to test for the presence or
absence of NULL:
SELECT *
FROM mytable
WHERE column1 <> column2
OR column1 IS NULL
OR column2 IS NULL
You can read about three-value logic and NULL values in Books Online:
http://msdn.microsoft.com/library/e..._qd_02_8pwy.asp
In general a comparison of anything to NULL is always UNKNOWN which means
NULL values are excluded by statements like
...
WHERE x <> y
David Portas
--
Please reply only to the newsgroup
--|||ANSI defines NULL = NULL as false.
NULL is the abscence of a value and, NULL does not equal NULL.
You can:
a) use IS NULL to perform the test or
b) you could say SET ANSI_NULLS OFF
Books Online will have lot's more info on each option... I hope this helps,
Brian Moran
Principal Mentor
Solid Quality Learning
SQL Server MVP
http://www.solidqualitylearning.com
"James" <anonymous@.discussions.microsoft.com> wrote in message
news:000e01c3dc43$7c50b750$a101280a@.phx.gbl...
quote:

> I have two Smalldatetime columns. One of the columns have
> NULL values in some of the rows. When I compare the two
> columns, I don't get the rows that have NULL values (I am
> comparing them to be NOT the same e.g Select * from
> mytable where column1 <> column2). I get all the rows that
> are not the same but have values but not the NULL columns.
> How can I get the NULL columns too '
>
> Thanks for help.
|||> b) you could say SET ANSI_NULLS OFF
ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE - it's still UNKNOWN.
David Portas
--
Please reply only to the newsgroup
--|||I guess I have something wrong in my query. I also was
trying to exclude the values in one of the columns which
was the dates that had 1900 in it (E.g NOT Like '%1900%').
I was trying to add that statement to
SELECT *
FROM mytable
WHERE column1 <> column2
OR column1 IS NULL
OR column2 IS NULL
--OR/AND column1 Not like '%1900%'
but somehow it is eliminating the NULL columns.
Thanks.
quote:

>--Original Message--
>ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE -

it's still UNKNOWN.
quote:

>--
>David Portas
>--
>Please reply only to the newsgroup
>--
>
>.
>
|||I guess I have something wrong in my query. I also was
trying to exclude the values in one of the columns which
was the dates that had 1900 in it (E.g NOT Like '%1900%').
I was trying to add that statement to
SELECT *
FROM mytable
WHERE column1 <> column2
OR column1 IS NULL
OR column2 IS NULL
--OR/AND column1 Not like '%1900%'
but somehow it is eliminating the NULL columns.
Thanks.
quote:

>--Original Message--
>ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE -

it's still UNKNOWN.
quote:

>--
>David Portas
>--
>Please reply only to the newsgroup
>--
>
>.
>
|||SORRY FOR THE DOUBLE POST...
quote:

>--Original Message--
>I guess I have something wrong in my query. I also was
>trying to exclude the values in one of the columns which
>was the dates that had 1900 in it (E.g NOT Like '%

1900%').
quote:

>I was trying to add that statement to
>SELECT *
> FROM mytable
> WHERE column1 <> column2
> OR column1 IS NULL
> OR column2 IS NULL
>--OR/AND column1 Not like '%1900%'
>but somehow it is eliminating the NULL columns.
>Thanks.
>
>it's still UNKNOWN.
>.
>
|||It helps if you can post some code to reproduce your problem and show your
required results. It's not clear to me exactly what result you're trying to
get in this case.
When combining AND and OR you have to watch out for the order of precedence.
Use parentheses to make the order of evaluation clear otherwise x OR y AND z
is evaluated as x OR (y AND z).
Don't use LIKE to compare dates. LIKE performs a string comparison and if
you use it on dates then every date has to be cast as a string. Best way to
eliminate the year 1900 is to test for a date >= 1901-01-01.
Here's some code to reproduce a result that *may* be what you want:
CREATE TABLE MyTable (keycol INTEGER PRIMARY KEY, column1 SMALLDATETIME,
column2 SMALLDATETIME)
INSERT INTO MyTable VALUES (1,'20030101','20030101')
INSERT INTO MyTable VALUES (2,'20030101','20030102')
INSERT INTO MyTable VALUES (3,'19000101','19000101')
INSERT INTO MyTable VALUES (4,'19000101','20030101')
INSERT INTO MyTable VALUES (5,'20030101','19000101')
INSERT INTO MyTable VALUES (6,'19000101',NULL)
INSERT INTO MyTable VALUES (7,NULL,'19000101')
INSERT INTO MyTable VALUES (8,'20030101',NULL)
INSERT INTO MyTable VALUES (9,NULL,'20030101')
INSERT INTO MyTable VALUES (10,NULL,NULL)
SELECT *
FROM MyTable
WHERE
COALESCE(column1,'19000101') <> COALESCE(column2,'19000101')
AND
(column1 >= '19010101' OR column2 >= '19010101')
Which returns the rows where Keycol is 2,4,5,8 and 9.
If that's not it and you need more help then you'll have to specify which
rows you want to be included in your result. I should have covered all the
relevant combinations.
Hope this helps.
David Portas
--
Please reply only to the newsgroup
--|||Thanks a lot..........
James.
quote:

>--Original Message--
>It helps if you can post some code to reproduce your

problem and show your
quote:

>required results. It's not clear to me exactly what

result you're trying to
quote:

>get in this case.
>When combining AND and OR you have to watch out for the

order of precedence.
quote:

>Use parentheses to make the order of evaluation clear

otherwise x OR y AND z
quote:

>is evaluated as x OR (y AND z).
>Don't use LIKE to compare dates. LIKE performs a string

comparison and if
quote:

>you use it on dates then every date has to be cast as a

string. Best way to
quote:

>eliminate the year 1900 is to test for a date >= 1901-01-

01.
quote:

>Here's some code to reproduce a result that *may* be what

you want:
quote:

>CREATE TABLE MyTable (keycol INTEGER PRIMARY KEY, column1

SMALLDATETIME,
quote:

>column2 SMALLDATETIME)
>INSERT INTO MyTable VALUES (1,'20030101','20030101')
>INSERT INTO MyTable VALUES (2,'20030101','20030102')
>INSERT INTO MyTable VALUES (3,'19000101','19000101')
>INSERT INTO MyTable VALUES (4,'19000101','20030101')
>INSERT INTO MyTable VALUES (5,'20030101','19000101')
>INSERT INTO MyTable VALUES (6,'19000101',NULL)
>INSERT INTO MyTable VALUES (7,NULL,'19000101')
>INSERT INTO MyTable VALUES (8,'20030101',NULL)
>INSERT INTO MyTable VALUES (9,NULL,'20030101')
>INSERT INTO MyTable VALUES (10,NULL,NULL)
>SELECT *
> FROM MyTable
> WHERE
> COALESCE(column1,'19000101') <> COALESCE

(column2,'19000101')
quote:
d">
> AND
> (column1 >= '19010101' OR column2 >= '19010101')
>Which returns the rows where Keycol is 2,4,5,8 and 9.
>If that's not it and you need more help then you'll have

to specify which
quote:

>rows you want to be included in your result. I should

have covered all the
quote:

>relevant combinations.
>Hope this helps.
>--
>David Portas
>--
>Please reply only to the newsgroup
>--
>
>.
>

Comparing Smalldatetime column

I have two Smalldatetime columns. One of the columns have
NULL values in some of the rows. When I compare the two
columns, I don't get the rows that have NULL values (I am
comparing them to be NOT the same e.g Select * from
mytable where column1 <> column2). I get all the rows that
are not the same but have values but not the NULL columns.
How can I get the NULL columns too '
Thanks for help.IS NULL and IS NOT NULL are the predicates used to test for the presence or
absence of NULL:
SELECT *
FROM mytable
WHERE column1 <> column2
OR column1 IS NULL
OR column2 IS NULL
You can read about three-value logic and NULL values in Books Online:
http://msdn.microsoft.com/library/en-us/acdata/ac_8_qd_02_8pwy.asp
In general a comparison of anything to NULL is always UNKNOWN which means
NULL values are excluded by statements like
...
WHERE x <> y
--
David Portas
--
Please reply only to the newsgroup
--|||ANSI defines NULL = NULL as false.
NULL is the abscence of a value and, NULL does not equal NULL.
You can:
a) use IS NULL to perform the test or
b) you could say SET ANSI_NULLS OFF
Books Online will have lot's more info on each option... I hope this helps,
--
Brian Moran
Principal Mentor
Solid Quality Learning
SQL Server MVP
http://www.solidqualitylearning.com
"James" <anonymous@.discussions.microsoft.com> wrote in message
news:000e01c3dc43$7c50b750$a101280a@.phx.gbl...
> I have two Smalldatetime columns. One of the columns have
> NULL values in some of the rows. When I compare the two
> columns, I don't get the rows that have NULL values (I am
> comparing them to be NOT the same e.g Select * from
> mytable where column1 <> column2). I get all the rows that
> are not the same but have values but not the NULL columns.
> How can I get the NULL columns too '
>
> Thanks for help.|||> b) you could say SET ANSI_NULLS OFF
ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE - it's still UNKNOWN.
--
David Portas
--
Please reply only to the newsgroup
--|||I guess I have something wrong in my query. I also was
trying to exclude the values in one of the columns which
was the dates that had 1900 in it (E.g NOT Like '%1900%').
I was trying to add that statement to
SELECT *
FROM mytable
WHERE column1 <> column2
OR column1 IS NULL
OR column2 IS NULL
--OR/AND column1 Not like '%1900%'
but somehow it is eliminating the NULL columns.
Thanks.
>--Original Message--
>> b) you could say SET ANSI_NULLS OFF
>ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE -
it's still UNKNOWN.
>--
>David Portas
>--
>Please reply only to the newsgroup
>--
>
>.
>|||I guess I have something wrong in my query. I also was
trying to exclude the values in one of the columns which
was the dates that had 1900 in it (E.g NOT Like '%1900%').
I was trying to add that statement to
SELECT *
FROM mytable
WHERE column1 <> column2
OR column1 IS NULL
OR column2 IS NULL
--OR/AND column1 Not like '%1900%'
but somehow it is eliminating the NULL columns.
Thanks.
>--Original Message--
>> b) you could say SET ANSI_NULLS OFF
>ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE -
it's still UNKNOWN.
>--
>David Portas
>--
>Please reply only to the newsgroup
>--
>
>.
>|||SORRY FOR THE DOUBLE POST...
>--Original Message--
>I guess I have something wrong in my query. I also was
>trying to exclude the values in one of the columns which
>was the dates that had 1900 in it (E.g NOT Like '%
1900%').
>I was trying to add that statement to
>SELECT *
> FROM mytable
> WHERE column1 <> column2
> OR column1 IS NULL
> OR column2 IS NULL
>--OR/AND column1 Not like '%1900%'
>but somehow it is eliminating the NULL columns.
>Thanks.
>>--Original Message--
>> b) you could say SET ANSI_NULLS OFF
>>ANSI_NULLS OFF won't evaluate x<>NULL to TRUE or FALSE -
>it's still UNKNOWN.
>>--
>>David Portas
>>--
>>Please reply only to the newsgroup
>>--
>>
>>.
>.
>|||It helps if you can post some code to reproduce your problem and show your
required results. It's not clear to me exactly what result you're trying to
get in this case.
When combining AND and OR you have to watch out for the order of precedence.
Use parentheses to make the order of evaluation clear otherwise x OR y AND z
is evaluated as x OR (y AND z).
Don't use LIKE to compare dates. LIKE performs a string comparison and if
you use it on dates then every date has to be cast as a string. Best way to
eliminate the year 1900 is to test for a date >= 1901-01-01.
Here's some code to reproduce a result that *may* be what you want:
CREATE TABLE MyTable (keycol INTEGER PRIMARY KEY, column1 SMALLDATETIME,
column2 SMALLDATETIME)
INSERT INTO MyTable VALUES (1,'20030101','20030101')
INSERT INTO MyTable VALUES (2,'20030101','20030102')
INSERT INTO MyTable VALUES (3,'19000101','19000101')
INSERT INTO MyTable VALUES (4,'19000101','20030101')
INSERT INTO MyTable VALUES (5,'20030101','19000101')
INSERT INTO MyTable VALUES (6,'19000101',NULL)
INSERT INTO MyTable VALUES (7,NULL,'19000101')
INSERT INTO MyTable VALUES (8,'20030101',NULL)
INSERT INTO MyTable VALUES (9,NULL,'20030101')
INSERT INTO MyTable VALUES (10,NULL,NULL)
SELECT *
FROM MyTable
WHERE
COALESCE(column1,'19000101') <> COALESCE(column2,'19000101')
AND
(column1 >= '19010101' OR column2 >= '19010101')
Which returns the rows where Keycol is 2,4,5,8 and 9.
If that's not it and you need more help then you'll have to specify which
rows you want to be included in your result. I should have covered all the
relevant combinations.
Hope this helps.
--
David Portas
--
Please reply only to the newsgroup
--|||Thanks a lot..........
James.
>--Original Message--
>It helps if you can post some code to reproduce your
problem and show your
>required results. It's not clear to me exactly what
result you're trying to
>get in this case.
>When combining AND and OR you have to watch out for the
order of precedence.
>Use parentheses to make the order of evaluation clear
otherwise x OR y AND z
>is evaluated as x OR (y AND z).
>Don't use LIKE to compare dates. LIKE performs a string
comparison and if
>you use it on dates then every date has to be cast as a
string. Best way to
>eliminate the year 1900 is to test for a date >= 1901-01-
01.
>Here's some code to reproduce a result that *may* be what
you want:
>CREATE TABLE MyTable (keycol INTEGER PRIMARY KEY, column1
SMALLDATETIME,
>column2 SMALLDATETIME)
>INSERT INTO MyTable VALUES (1,'20030101','20030101')
>INSERT INTO MyTable VALUES (2,'20030101','20030102')
>INSERT INTO MyTable VALUES (3,'19000101','19000101')
>INSERT INTO MyTable VALUES (4,'19000101','20030101')
>INSERT INTO MyTable VALUES (5,'20030101','19000101')
>INSERT INTO MyTable VALUES (6,'19000101',NULL)
>INSERT INTO MyTable VALUES (7,NULL,'19000101')
>INSERT INTO MyTable VALUES (8,'20030101',NULL)
>INSERT INTO MyTable VALUES (9,NULL,'20030101')
>INSERT INTO MyTable VALUES (10,NULL,NULL)
>SELECT *
> FROM MyTable
> WHERE
> COALESCE(column1,'19000101') <> COALESCE
(column2,'19000101')
> AND
> (column1 >= '19010101' OR column2 >= '19010101')
>Which returns the rows where Keycol is 2,4,5,8 and 9.
>If that's not it and you need more help then you'll have
to specify which
>rows you want to be included in your result. I should
have covered all the
>relevant combinations.
>Hope this helps.
>--
>David Portas
>--
>Please reply only to the newsgroup
>--
>
>.
>

Comparing Rows in a table

Hi guys,

I just wanted to know if it is possible to compare each row in the table with the previous one.

Im looking to compare each row the time column with the previous to see if there is a gap of more then 15 minutes.

I was thinking of using the datediff function to compare but not sure how i go about accessing row by row and compare with the previous one.

Any help much appreciated.

Thanks.

I would suggest using a self-join based on your datetime field. For this kind of join it is critical that you have an index built on the target datetime field. I will put together a mock-up of this.|||

I dont understand target datetime field?

Do you mean the table that ill be out putting the results to?

Thanks for your quick response and putting together the mock-up of this.

|||

Here is a mockup. I sprayed 32767 records out over 43 days going backwards from 3/20/7 using an iterator table and a scalar RAND udf. These two items can be found on this page:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1330536&SiteID=1

Here is the mockup. I have not yet evaluated whether or not this is efficient. Stand by; another post will follow..

declare @.mockup table
( rowid integer,
aDateTime datetime,
unique (aDateTime, rowId)
)

insert into @.mockup
select iter,
cast ('3/20/7' as datetime) - 43 * dbo.rand() as aDateTime
from small_iterator
--select * from @.mockup

select rowId,
aDateTime,
nextDateTime,
convert (varchar(12), nextDateTime - aDateTime, 108)
as [Time Difference]
from ( select rowId,
aDatetime,
( select top 1 b.aDateTime
from @.mockup b
where b.aDateTime > a.aDateTime
order by aDateTime, rowId
) as nextDatetime
from @.mockup a
) x
where nextDateTime - aDateTime >= cast(cast ('0:15:00' as datetime) as float)

-- rowId aDateTime nextDateTime
-- - -- --
-- 23623 2007-02-11 09:51:57.530 2007-02-11 10:07:25.077 00:15:27
-- 12068 2007-02-12 11:46:03.190 2007-02-12 12:01:25.987 00:15:22
-- 14377 2007-02-14 03:17:02.890 2007-02-14 03:32:39.907 00:15:37
-- ...

-- 30564 2007-03-17 04:32:18.287 2007-03-17 04:48:51.520 00:16:33
-- 9053 2007-03-18 21:07:15.663 2007-03-18 21:22:21.220 00:15:05

Oh yeah: I didn't perform a self join; this looked better. Sorry, I am in the middle of a restore.

OK! This puppy doesn't look too bad: With this index I am getting a plan of index scan / index seek. To process the entire 32767 records, the IO is:

-- Table '#401BB779'. Scan count 32768, logical reads 65746, physical reads 0, read-ahead reads 0.

That's pretty good. You might be able to find a better way to do this, but this is at least pretty good.

REMEMBER!

It is CRITICAL that the date/time column has an index! Otherwise you are NOT going to be able to search by index seek.

|||

Just to clarify - how do you define your previous row? Is your definition based on the primary key, or the datetime column?

i.e.

ID Date

1 05-Jun-2007

2 06-Mar-2007

3 02-Dec-2006

In the above example - if the current row is row 2, are you looking to compare row 2's date with that of row 1 or row 3?

Chris

|||

In our GPS Table we have an ID column for each entry.

What we need to do is compare current row time by next row time.

For example

in you example above we would compare Row 1 with Row 2 if there is a gap of 15 mins in time we record that.

Else we move to row 2 and compare it with row 3 and see if there is a gap of 15 minutes.

We need to do this for our entire database table and record each time there is gap of 15 mins or greater between 2 entries in the table.

Our goal is to split the table into a new trips table.

|||

Does your table have an IDENTITY column? If so, are there any gaps between the IDENTITY values?

Thanks
Chris

|||

Thanks for your reply.

The date time column is only referenced by the Auto ID that is created when the data is inputted into our GPS Table.

We need to compare row 1 and with row 2 and check if there is a 15 minute gap or greater in time.

If there is we want to record this in our Trip table.

We would like to do this for the entire table and record each time there is a gap of 15 minutes between the current row and the next row.

|||

Yes our table has an IDENTITY column. This ID is created everytime data is inputted into our GPS table.

This column will never be blank as the number is generated when an entry is made to the table.

|||Hey all!
Thanks for your Class I help so far.
I'm involved in this project too.. The data comes in something like this:

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
338 212 2006-1-12 17:23:33
339 212 2006-1-12 17:23:58
340 212 2006-1-12 17:24:33|||

Previous post removed as I've found a solution that can cope with gaps between values in the IDENTITY column. See below.

Chris

--Setup test data
DECLARE @.MyTable TABLE (ID INT NOT NULL, VehicleID INT, [Date/Time] DATETIME NOT NULL)

INSERT INTO @.MyTable(ID, VehicleID, [Date/Time])
SELECT 333, 212, '2006-1-12 10:45:33' UNION
SELECT 334, 212, '2006-1-12 10:45:48' UNION
SELECT 335, 212, '2006-1-12 10:46:32' UNION
SELECT 336, 212, '2006-1-12 10:47:43' UNION
SELECT 337, 212, '2006-1-12 10:47:52' UNION
--<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
SELECT 338, 212, '2006-1-12 17:23:33' UNION
SELECT 339, 212, '2006-1-12 17:23:58' UNION
SELECT 340, 212, '2006-1-12 17:24:33' UNION
--Repeat data for a different VehicleID
SELECT 341, 100, '2006-1-12 10:45:33' UNION
SELECT 342, 100, '2006-1-12 10:45:48' UNION
SELECT 343, 100, '2006-1-12 10:46:32' UNION
SELECT 344, 100, '2006-1-12 10:47:43' UNION
SELECT 345, 100, '2006-1-12 10:47:52' UNION
--<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
SELECT 346, 100, '2006-1-12 17:23:33' UNION
SELECT 347, 100, '2006-1-12 17:23:58' UNION
SELECT 348, 100, '2006-1-12 17:24:33'

SELECT *
FROM @.MyTable

--Return the rows between which there is a datetime gap > 15 mins
--All columns are returned purely for this example - take out any columns you don't need and add in any that you do
SELECT t1.ID, t1.[Date/Time],
t2.ID,
t2.[Date/Time],
DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time]) AS [Difference (mins)],
t1.VehicleID
FROM @.MyTable t1, @.MyTable t2
WHERE t2.[Date/Time] > DATEADD(mi, 15, t1.[Date/Time])
AND t2.ID = (SELECT TOP 1 t3.ID FROM @.MyTable t3 WHERE t3.ID > t1.ID AND t3.VehicleID = t1.VehicleID ORDER BY t3.ID ASC)
AND t2.VehicleID = t1.VehicleID

--Delete a couple of rows to show we can cope with range gaps
DELETE FROM @.MyTable
WHERE id IN (338, 345)

--Repeat the query just to show we can cope with range gaps, expect different results as rows have been deleted from the source table
SELECT t1.ID, t1.[Date/Time],
t2.ID,
t2.[Date/Time],
DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time]) AS [Difference (mins)],
t1.VehicleID
FROM @.MyTable t1, @.MyTable t2
WHERE t2.[Date/Time] > DATEADD(mi, 15, t1.[Date/Time])
AND t2.ID = (SELECT TOP 1 t3.ID FROM @.MyTable t3 WHERE t3.ID > t1.ID AND t3.VehicleID = t1.VehicleID ORDER BY t3.ID ASC)
AND t2.VehicleID = t1.VehicleID

|||

Hi Chris,

Thanks for your response.

I cant test it out until tomorrow as i dont have access to the database table from home.

I will try this out tomorrow and keep you informed.

Thanks a mil.

|||

Emerson, Carrics3,

How about (sorry for the double spacing, which I don't know how to control):

create table new_trips

(start_ID int not null,

vehicle_ID int,

start_time datetime

)

;

insert into new_trips

select

t2.ID,

t2.Vehicle_ID,

t2.[Date/Time]

from data_table t1

inner join data_table t2 -- you are joining a table to itself

on t2.ID = (t1.ID + 1) and

t2.vehicle_ID = t1.vehicle_ID

where DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time])

order by 1, 2

;

Dan

P.S. This does not properly deal with having multiple Vehicle_ID in the same table. I will have to think a bit about how best to deal with that. It may involve placing ID and Vehicle_ID in a sorted fashion into a temporary table, and having an additional column (an IDENTITY column) in the temporary table that would be used the same way ID is being used in the above query. When Vehicle_ID changed it should also indicate a new trip -- I am guessing, anyway.

|||

In response to DanR1's post, just be aware that joining onto an identity value + 1 (and incorporating the VehicleID) will fail where either the VehicleID changes between adjacent rows or where there are gaps between identity values (where rows have been deleted or transactions containing inserts not committed).

If your data looks like either of the two examples below then this method will fail.

Chris

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
338 213 2006-1-12 17:23:33
339 212 2006-1-12 17:23:35
340 212 2006-1-12 17:23:58
341 212 2006-1-12 17:24:33

or

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
339 212 2006-1-12 17:23:33
340 212 2006-1-12 17:23:35
341 212 2006-1-12 17:23:58
342 212 2006-1-12 17:24:33

|||

Hi Chris,

I have tested out your sql statement.

It is working but its not exactly what we want to do.

We need to compare row 1 with row 2 and check if there is a gap of 15 mins

If there is we want to record that ( Instead of Checking row 1 against every record in the table)

However if there is a gap of less then 15 mins then we want to move on and query row 2 with row 3 and check if there is gap of 15 mins. And so on til we get to the end where there is no row coming after the current row.

Each time there is gap of 15 mins or greater between the current row and the next row we want to record it in our Trip table

ID VehicleID Date/Time
333 353864504523 2006-1-12 10:35:33
334 353864504523 2006-1-12 10:35:48
335 353864504523 2006-1-12 10:36:32
336 353864504523 2006-1-12 10:37:43
337 353864504523 2006-1-12 10:38:52
338 353864504523 2006-1-12 10:39:33
339 353864504523 2006-1-12 10:40:48 There is a gap of 15 mins between these 2 entries.
340 353864504523 2006-1-12 10:55:32
341 353864504523 2006-1-12 10:55:43
342 353864504523 2006-1-12 10:56:52

Comparing Rows in a table

Hi guys,

I just wanted to know if it is possible to compare each row in the table with the previous one.

Im looking to compare each row the time column with the previous to see if there is a gap of more then 15 minutes.

I was thinking of using the datediff function to compare but not sure how i go about accessing row by row and compare with the previous one.

Any help much appreciated.

Thanks.

I would suggest using a self-join based on your datetime field. For this kind of join it is critical that you have an index built on the target datetime field. I will put together a mock-up of this.|||

I dont understand target datetime field?

Do you mean the table that ill be out putting the results to?

Thanks for your quick response and putting together the mock-up of this.

|||

Here is a mockup. I sprayed 32767 records out over 43 days going backwards from 3/20/7 using an iterator table and a scalar RAND udf. These two items can be found on this page:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1330536&SiteID=1

Here is the mockup. I have not yet evaluated whether or not this is efficient. Stand by; another post will follow..

declare @.mockup table
( rowid integer,
aDateTime datetime,
unique (aDateTime, rowId)
)

insert into @.mockup
select iter,
cast ('3/20/7' as datetime) - 43 * dbo.rand() as aDateTime
from small_iterator
--select * from @.mockup

select rowId,
aDateTime,
nextDateTime,
convert (varchar(12), nextDateTime - aDateTime, 108)
as [Time Difference]
from ( select rowId,
aDatetime,
( select top 1 b.aDateTime
from @.mockup b
where b.aDateTime > a.aDateTime
order by aDateTime, rowId
) as nextDatetime
from @.mockup a
) x
where nextDateTime - aDateTime >= cast(cast ('0:15:00' as datetime) as float)

-- rowId aDateTime nextDateTime
-- - -- --
-- 23623 2007-02-11 09:51:57.530 2007-02-11 10:07:25.077 00:15:27
-- 12068 2007-02-12 11:46:03.190 2007-02-12 12:01:25.987 00:15:22
-- 14377 2007-02-14 03:17:02.890 2007-02-14 03:32:39.907 00:15:37
-- ...

-- 30564 2007-03-17 04:32:18.287 2007-03-17 04:48:51.520 00:16:33
-- 9053 2007-03-18 21:07:15.663 2007-03-18 21:22:21.220 00:15:05

Oh yeah: I didn't perform a self join; this looked better. Sorry, I am in the middle of a restore.

OK! This puppy doesn't look too bad: With this index I am getting a plan of index scan / index seek. To process the entire 32767 records, the IO is:

-- Table '#401BB779'. Scan count 32768, logical reads 65746, physical reads 0, read-ahead reads 0.

That's pretty good. You might be able to find a better way to do this, but this is at least pretty good.

REMEMBER!

It is CRITICAL that the date/time column has an index! Otherwise you are NOT going to be able to search by index seek.

|||

Just to clarify - how do you define your previous row? Is your definition based on the primary key, or the datetime column?

i.e.

ID Date

1 05-Jun-2007

2 06-Mar-2007

3 02-Dec-2006

In the above example - if the current row is row 2, are you looking to compare row 2's date with that of row 1 or row 3?

Chris

|||

In our GPS Table we have an ID column for each entry.

What we need to do is compare current row time by next row time.

For example

in you example above we would compare Row 1 with Row 2 if there is a gap of 15 mins in time we record that.

Else we move to row 2 and compare it with row 3 and see if there is a gap of 15 minutes.

We need to do this for our entire database table and record each time there is gap of 15 mins or greater between 2 entries in the table.

Our goal is to split the table into a new trips table.

|||

Does your table have an IDENTITY column? If so, are there any gaps between the IDENTITY values?

Thanks
Chris

|||

Thanks for your reply.

The date time column is only referenced by the Auto ID that is created when the data is inputted into our GPS Table.

We need to compare row 1 and with row 2 and check if there is a 15 minute gap or greater in time.

If there is we want to record this in our Trip table.

We would like to do this for the entire table and record each time there is a gap of 15 minutes between the current row and the next row.

|||

Yes our table has an IDENTITY column. This ID is created everytime data is inputted into our GPS table.

This column will never be blank as the number is generated when an entry is made to the table.

|||Hey all!
Thanks for your Class I help so far.
I'm involved in this project too.. The data comes in something like this:

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
338 212 2006-1-12 17:23:33
339 212 2006-1-12 17:23:58
340 212 2006-1-12 17:24:33
|||

Previous post removed as I've found a solution that can cope with gaps between values in the IDENTITY column. See below.

Chris

--Setup test data
DECLARE @.MyTable TABLE (ID INT NOT NULL, VehicleID INT, [Date/Time] DATETIME NOT NULL)

INSERT INTO @.MyTable(ID, VehicleID, [Date/Time])
SELECT 333, 212, '2006-1-12 10:45:33' UNION
SELECT 334, 212, '2006-1-12 10:45:48' UNION
SELECT 335, 212, '2006-1-12 10:46:32' UNION
SELECT 336, 212, '2006-1-12 10:47:43' UNION
SELECT 337, 212, '2006-1-12 10:47:52' UNION
--<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
SELECT 338, 212, '2006-1-12 17:23:33' UNION
SELECT 339, 212, '2006-1-12 17:23:58' UNION
SELECT 340, 212, '2006-1-12 17:24:33' UNION
--Repeat data for a different VehicleID
SELECT 341, 100, '2006-1-12 10:45:33' UNION
SELECT 342, 100, '2006-1-12 10:45:48' UNION
SELECT 343, 100, '2006-1-12 10:46:32' UNION
SELECT 344, 100, '2006-1-12 10:47:43' UNION
SELECT 345, 100, '2006-1-12 10:47:52' UNION
--<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
SELECT 346, 100, '2006-1-12 17:23:33' UNION
SELECT 347, 100, '2006-1-12 17:23:58' UNION
SELECT 348, 100, '2006-1-12 17:24:33'

SELECT *
FROM @.MyTable

--Return the rows between which there is a datetime gap > 15 mins
--All columns are returned purely for this example - take out any columns you don't need and add in any that you do
SELECT t1.ID, t1.[Date/Time],
t2.ID,
t2.[Date/Time],
DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time]) AS [Difference (mins)],
t1.VehicleID
FROM @.MyTable t1, @.MyTable t2
WHERE t2.[Date/Time] > DATEADD(mi, 15, t1.[Date/Time])
AND t2.ID = (SELECT TOP 1 t3.ID FROM @.MyTable t3 WHERE t3.ID > t1.ID AND t3.VehicleID = t1.VehicleID ORDER BY t3.ID ASC)
AND t2.VehicleID = t1.VehicleID

--Delete a couple of rows to show we can cope with range gaps
DELETE FROM @.MyTable
WHERE id IN (338, 345)

--Repeat the query just to show we can cope with range gaps, expect different results as rows have been deleted from the source table
SELECT t1.ID, t1.[Date/Time],
t2.ID,
t2.[Date/Time],
DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time]) AS [Difference (mins)],
t1.VehicleID
FROM @.MyTable t1, @.MyTable t2
WHERE t2.[Date/Time] > DATEADD(mi, 15, t1.[Date/Time])
AND t2.ID = (SELECT TOP 1 t3.ID FROM @.MyTable t3 WHERE t3.ID > t1.ID AND t3.VehicleID = t1.VehicleID ORDER BY t3.ID ASC)
AND t2.VehicleID = t1.VehicleID

|||

Hi Chris,

Thanks for your response.

I cant test it out until tomorrow as i dont have access to the database table from home.

I will try this out tomorrow and keep you informed.

Thanks a mil.

|||

Emerson, Carrics3,

How about (sorry for the double spacing, which I don't know how to control):

create table new_trips

(start_ID int not null,

vehicle_ID int,

start_time datetime

)

;

insert into new_trips

select

t2.ID,

t2.Vehicle_ID,

t2.[Date/Time]

from data_table t1

inner join data_table t2 -- you are joining a table to itself

on t2.ID = (t1.ID + 1) and

t2.vehicle_ID = t1.vehicle_ID

where DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time])

order by 1, 2

;

Dan

P.S. This does not properly deal with having multiple Vehicle_ID in the same table. I will have to think a bit about how best to deal with that. It may involve placing ID and Vehicle_ID in a sorted fashion into a temporary table, and having an additional column (an IDENTITY column) in the temporary table that would be used the same way ID is being used in the above query. When Vehicle_ID changed it should also indicate a new trip -- I am guessing, anyway.

|||

In response to DanR1's post, just be aware that joining onto an identity value + 1 (and incorporating the VehicleID) will fail where either the VehicleID changes between adjacent rows or where there are gaps between identity values (where rows have been deleted or transactions containing inserts not committed).

If your data looks like either of the two examples below then this method will fail.

Chris

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
338 213 2006-1-12 17:23:33
339 212 2006-1-12 17:23:35
340 212 2006-1-12 17:23:58
341 212 2006-1-12 17:24:33

or

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
339 212 2006-1-12 17:23:33
340 212 2006-1-12 17:23:35
341 212 2006-1-12 17:23:58
342 212 2006-1-12 17:24:33

|||

Hi Chris,

I have tested out your sql statement.

It is working but its not exactly what we want to do.

We need to compare row 1 with row 2 and check if there is a gap of 15 mins

If there is we want to record that ( Instead of Checking row 1 against every record in the table)

However if there is a gap of less then 15 mins then we want to move on and query row 2 with row 3 and check if there is gap of 15 mins. And so on til we get to the end where there is no row coming after the current row.

Each time there is gap of 15 mins or greater between the current row and the next row we want to record it in our Trip table

ID VehicleID Date/Time
333 353864504523 2006-1-12 10:35:33
334 353864504523 2006-1-12 10:35:48
335 353864504523 2006-1-12 10:36:32
336 353864504523 2006-1-12 10:37:43
337 353864504523 2006-1-12 10:38:52
338 353864504523 2006-1-12 10:39:33
339 353864504523 2006-1-12 10:40:48 There is a gap of 15 mins between these 2 entries.
340 353864504523 2006-1-12 10:55:32
341 353864504523 2006-1-12 10:55:43
342 353864504523 2006-1-12 10:56:52

sqlsql

Comparing Rows in a table

Hi guys,

I just wanted to know if it is possible to compare each row in the table with the previous one.

Im looking to compare each row the time column with the previous to see if there is a gap of more then 15 minutes.

I was thinking of using the datediff function to compare but not sure how i go about accessing row by row and compare with the previous one.

Any help much appreciated.

Thanks.

I would suggest using a self-join based on your datetime field. For this kind of join it is critical that you have an index built on the target datetime field. I will put together a mock-up of this.|||

I dont understand target datetime field?

Do you mean the table that ill be out putting the results to?

Thanks for your quick response and putting together the mock-up of this.

|||

Here is a mockup. I sprayed 32767 records out over 43 days going backwards from 3/20/7 using an iterator table and a scalar RAND udf. These two items can be found on this page:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1330536&SiteID=1

Here is the mockup. I have not yet evaluated whether or not this is efficient. Stand by; another post will follow..

declare @.mockup table
( rowid integer,
aDateTime datetime,
unique (aDateTime, rowId)
)

insert into @.mockup
select iter,
cast ('3/20/7' as datetime) - 43 * dbo.rand() as aDateTime
from small_iterator
--select * from @.mockup

select rowId,
aDateTime,
nextDateTime,
convert (varchar(12), nextDateTime - aDateTime, 108)
as [Time Difference]
from ( select rowId,
aDatetime,
( select top 1 b.aDateTime
from @.mockup b
where b.aDateTime > a.aDateTime
order by aDateTime, rowId
) as nextDatetime
from @.mockup a
) x
where nextDateTime - aDateTime >= cast(cast ('0:15:00' as datetime) as float)

-- rowId aDateTime nextDateTime
-- - -- --
-- 23623 2007-02-11 09:51:57.530 2007-02-11 10:07:25.077 00:15:27
-- 12068 2007-02-12 11:46:03.190 2007-02-12 12:01:25.987 00:15:22
-- 14377 2007-02-14 03:17:02.890 2007-02-14 03:32:39.907 00:15:37
-- ...

-- 30564 2007-03-17 04:32:18.287 2007-03-17 04:48:51.520 00:16:33
-- 9053 2007-03-18 21:07:15.663 2007-03-18 21:22:21.220 00:15:05

Oh yeah: I didn't perform a self join; this looked better. Sorry, I am in the middle of a restore.

OK! This puppy doesn't look too bad: With this index I am getting a plan of index scan / index seek. To process the entire 32767 records, the IO is:

-- Table '#401BB779'. Scan count 32768, logical reads 65746, physical reads 0, read-ahead reads 0.

That's pretty good. You might be able to find a better way to do this, but this is at least pretty good.

REMEMBER!

It is CRITICAL that the date/time column has an index! Otherwise you are NOT going to be able to search by index seek.

|||

Just to clarify - how do you define your previous row? Is your definition based on the primary key, or the datetime column?

i.e.

ID Date

1 05-Jun-2007

2 06-Mar-2007

3 02-Dec-2006

In the above example - if the current row is row 2, are you looking to compare row 2's date with that of row 1 or row 3?

Chris

|||

In our GPS Table we have an ID column for each entry.

What we need to do is compare current row time by next row time.

For example

in you example above we would compare Row 1 with Row 2 if there is a gap of 15 mins in time we record that.

Else we move to row 2 and compare it with row 3 and see if there is a gap of 15 minutes.

We need to do this for our entire database table and record each time there is gap of 15 mins or greater between 2 entries in the table.

Our goal is to split the table into a new trips table.

|||

Does your table have an IDENTITY column? If so, are there any gaps between the IDENTITY values?

Thanks
Chris

|||

Thanks for your reply.

The date time column is only referenced by the Auto ID that is created when the data is inputted into our GPS Table.

We need to compare row 1 and with row 2 and check if there is a 15 minute gap or greater in time.

If there is we want to record this in our Trip table.

We would like to do this for the entire table and record each time there is a gap of 15 minutes between the current row and the next row.

|||

Yes our table has an IDENTITY column. This ID is created everytime data is inputted into our GPS table.

This column will never be blank as the number is generated when an entry is made to the table.

|||Hey all!
Thanks for your Class I help so far.
I'm involved in this project too.. The data comes in something like this:

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
338 212 2006-1-12 17:23:33
339 212 2006-1-12 17:23:58
340 212 2006-1-12 17:24:33
|||

Previous post removed as I've found a solution that can cope with gaps between values in the IDENTITY column. See below.

Chris

--Setup test data
DECLARE @.MyTable TABLE (ID INT NOT NULL, VehicleID INT, [Date/Time] DATETIME NOT NULL)

INSERT INTO @.MyTable(ID, VehicleID, [Date/Time])
SELECT 333, 212, '2006-1-12 10:45:33' UNION
SELECT 334, 212, '2006-1-12 10:45:48' UNION
SELECT 335, 212, '2006-1-12 10:46:32' UNION
SELECT 336, 212, '2006-1-12 10:47:43' UNION
SELECT 337, 212, '2006-1-12 10:47:52' UNION
--<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
SELECT 338, 212, '2006-1-12 17:23:33' UNION
SELECT 339, 212, '2006-1-12 17:23:58' UNION
SELECT 340, 212, '2006-1-12 17:24:33' UNION
--Repeat data for a different VehicleID
SELECT 341, 100, '2006-1-12 10:45:33' UNION
SELECT 342, 100, '2006-1-12 10:45:48' UNION
SELECT 343, 100, '2006-1-12 10:46:32' UNION
SELECT 344, 100, '2006-1-12 10:47:43' UNION
SELECT 345, 100, '2006-1-12 10:47:52' UNION
--<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
SELECT 346, 100, '2006-1-12 17:23:33' UNION
SELECT 347, 100, '2006-1-12 17:23:58' UNION
SELECT 348, 100, '2006-1-12 17:24:33'

SELECT *
FROM @.MyTable

--Return the rows between which there is a datetime gap > 15 mins
--All columns are returned purely for this example - take out any columns you don't need and add in any that you do
SELECT t1.ID, t1.[Date/Time],
t2.ID,
t2.[Date/Time],
DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time]) AS [Difference (mins)],
t1.VehicleID
FROM @.MyTable t1, @.MyTable t2
WHERE t2.[Date/Time] > DATEADD(mi, 15, t1.[Date/Time])
AND t2.ID = (SELECT TOP 1 t3.ID FROM @.MyTable t3 WHERE t3.ID > t1.ID AND t3.VehicleID = t1.VehicleID ORDER BY t3.ID ASC)
AND t2.VehicleID = t1.VehicleID

--Delete a couple of rows to show we can cope with range gaps
DELETE FROM @.MyTable
WHERE id IN (338, 345)

--Repeat the query just to show we can cope with range gaps, expect different results as rows have been deleted from the source table
SELECT t1.ID, t1.[Date/Time],
t2.ID,
t2.[Date/Time],
DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time]) AS [Difference (mins)],
t1.VehicleID
FROM @.MyTable t1, @.MyTable t2
WHERE t2.[Date/Time] > DATEADD(mi, 15, t1.[Date/Time])
AND t2.ID = (SELECT TOP 1 t3.ID FROM @.MyTable t3 WHERE t3.ID > t1.ID AND t3.VehicleID = t1.VehicleID ORDER BY t3.ID ASC)
AND t2.VehicleID = t1.VehicleID

|||

Hi Chris,

Thanks for your response.

I cant test it out until tomorrow as i dont have access to the database table from home.

I will try this out tomorrow and keep you informed.

Thanks a mil.

|||

Emerson, Carrics3,

How about (sorry for the double spacing, which I don't know how to control):

create table new_trips

(start_ID int not null,

vehicle_ID int,

start_time datetime

)

;

insert into new_trips

select

t2.ID,

t2.Vehicle_ID,

t2.[Date/Time]

from data_table t1

inner join data_table t2 -- you are joining a table to itself

on t2.ID = (t1.ID + 1) and

t2.vehicle_ID = t1.vehicle_ID

where DATEDIFF(mi, t1.[Date/Time], t2.[Date/Time])

order by 1, 2

;

Dan

P.S. This does not properly deal with having multiple Vehicle_ID in the same table. I will have to think a bit about how best to deal with that. It may involve placing ID and Vehicle_ID in a sorted fashion into a temporary table, and having an additional column (an IDENTITY column) in the temporary table that would be used the same way ID is being used in the above query. When Vehicle_ID changed it should also indicate a new trip -- I am guessing, anyway.

|||

In response to DanR1's post, just be aware that joining onto an identity value + 1 (and incorporating the VehicleID) will fail where either the VehicleID changes between adjacent rows or where there are gaps between identity values (where rows have been deleted or transactions containing inserts not committed).

If your data looks like either of the two examples below then this method will fail.

Chris

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
338 213 2006-1-12 17:23:33
339 212 2006-1-12 17:23:35
340 212 2006-1-12 17:23:58
341 212 2006-1-12 17:24:33

or

ID VehicleID Date/Time
333 212 2006-1-12 10:45:33
334 212 2006-1-12 10:45:48
335 212 2006-1-12 10:46:32
336 212 2006-1-12 10:47:43
337 212 2006-1-12 10:47:52
<< We want to identify and generate a new trip ID here in trip table (>15mins interval) >>
339 212 2006-1-12 17:23:33
340 212 2006-1-12 17:23:35
341 212 2006-1-12 17:23:58
342 212 2006-1-12 17:24:33

|||

Hi Chris,

I have tested out your sql statement.

It is working but its not exactly what we want to do.

We need to compare row 1 with row 2 and check if there is a gap of 15 mins

If there is we want to record that ( Instead of Checking row 1 against every record in the table)

However if there is a gap of less then 15 mins then we want to move on and query row 2 with row 3 and check if there is gap of 15 mins. And so on til we get to the end where there is no row coming after the current row.

Each time there is gap of 15 mins or greater between the current row and the next row we want to record it in our Trip table

ID VehicleID Date/Time
333 353864504523 2006-1-12 10:35:33
334 353864504523 2006-1-12 10:35:48
335 353864504523 2006-1-12 10:36:32
336 353864504523 2006-1-12 10:37:43
337 353864504523 2006-1-12 10:38:52
338 353864504523 2006-1-12 10:39:33
339 353864504523 2006-1-12 10:40:48 There is a gap of 15 mins between these 2 entries.
340 353864504523 2006-1-12 10:55:32
341 353864504523 2006-1-12 10:55:43
342 353864504523 2006-1-12 10:56:52

comparing rows

Is there any way of comparing specific rows in a table? i.e. I want to delete one of two rows in a table (that also has more rows in it other than the 2 i want to compare). Basicaly, each of the rows have a date field and a unique idtntifier, I want to compare two rows against each other using the unique indetifier and delete the one with the earliest date.

Anyone know how this can be done?

appreciated.......Please send sample script that create this two tables and inset couple of rows. This will be the best way to help you.

Eyal|||as the term unique identifier already states, it is unique. This means that you can't compare on that field. Isn't there another 'identifying' column (a foreign key)? Otherwise, if all columns except the identifier and date compare all columns to each other.

Sunday, March 25, 2012

comparing nvarchar(max) column using like to non-ASCII range

Our database defines the long_value column as nvarchar(max). I want to find out which rows actually contain non-ASCII characters in that column, but this clause also returns rows with only ASCII characters:

where long_value like (N'%[' + nchar(128) + N'-' + nchar(65535) + N']%')

What am I doing wrong?

It seems to be the fact that nchar(65535) has no length. I couldn't find any really good reference to cover this (perhaps this thread: http://groups.google.com/group/comp.lang.javascript/tree/browse_frm/month/2004-09/7d3603a75c1550f3?rnum=91&_done=%2Fgroup%2Fcomp.lang.javascript%2Fbrowse_frm%2Fmonth%2F2004-09%3F), but if you run these statements:

select N'%[' + nchar(128) + N'-' + nchar(65535) + N']%'

select N'%[' + nchar(128) + N'-' + nchar(65534) + N']%'

You will see that there is no second character after the - in the second string. This should do what you want:

declare @.long_value nvarchar(max)
set @.long_value = N'abcdefghijklmnopqrstuvwxyz' + nchar(129)
select case when @.long_value like (N'%[' + nchar(128) + N'-' + nchar(65534) + N']%') then 'yes' else 'no' end

and you could add a check for the nchar(65535) also...

|||

Louis Davidson wrote:

It seems to be the fact that nchar(65535) has no length. I couldn't find any really good reference to cover this (perhaps this thread: http://groups.google.com/group/comp.lang.javascript/tree/browse_frm/month/2004-09/7d3603a75c1550f3?rnum=91&_done=%2Fgroup%2Fcomp.lang.javascript%2Fbrowse_frm%2Fmonth%2F2004-09%3F), but if you run these statements:

select N'%[' + nchar(128) + N'-' + nchar(65535) + N']%'

select N'%[' + nchar(128) + N'-' + nchar(65534) + N']%'

You will see that there is no second character after the - in the second string.

I can't corroborate your results. I'm runing SQL Server 2005, both an unpatched Enterprise Edition and the SP1 + build 2153 hotfix Developer Edition. Connected to either server's master database, the result of both queries is displayed in Management Studio with an empty box on either side of the dash. Furthermore, the results of the following queries are 1, 2, 3, and 7 as expected:

select len(nchar(65535));

select len(nchar(128) + nchar(65535));

select len(nchar(128) + N'-' + nchar(65535));

select len(N'%[' + nchar(128) + N'-' + nchar(65535) + N']%');

Louis Davidson wrote:

This should do what you want:

declare @.long_value nvarchar(max)
set @.long_value = N'abcdefghijklmnopqrstuvwxyz' + nchar(129)
select case when @.long_value like (N'%[' + nchar(128) + N'-' + nchar(65534) + N']%') then 'yes' else 'no' end

and you could add a check for the nchar(65535) also...

Well, the obvious way to add a check for nchar(65535) is to include it in the range:

N'%[' + nchar(128) + N'-' + nchar(65534) + nchar(65535) + N']%'

Which, if it worked, would indicate a bug in character range matching. But it fails in the same way as the original.

Here is a diagnostic query that suggests to me that the problem has something to do with collations (a topic I had been blissfully ignorant of). If the server's default collation or the database's collation is Latin1_General_BIN2, it returns the expected result; but if the collation is SQL_Latin1_General_CP1_CI_AS, it returns 21,230 Unicode characters between 0 and 65535 inclusive which are equal to nchar(65535). Most disturbing is the fact that 0 (NULL) and 32 (SPACE) are among them, which might explain my original problem with the character range. Try this out on different servers and databases:

declare @.code as integer

set @.code = 0

while (@.code < 65536)

begin

if (nchar(@.code) = nchar(65535))

print cast(@.code as nchar) + ' equal'

set @.code = @.code + 1

end

|||

Kevin Rodgers wrote:

Here is a diagnostic query that suggests to me that the problem has something to do with collations (a topic I had been blissfully ignorant of). If the server's default collation or the database's collation is Latin1_General_BIN2, it returns the expected result; but if the collation is SQL_Latin1_General_CP1_CI_AS, it returns 21,230 Unicode characters between 0 and 65535 inclusive which are equal to nchar(65535). Most disturbing is the fact that 0 (NULL) and 32 (SPACE) are among them, which might explain my original problem with the character range. Try this out on different servers and databases:

declare @.code as integer

set @.code = 0

while (@.code < 65536)

begin

if (nchar(@.code) = nchar(65535))

print cast(@.code as nchar) + ' equal'

set @.code = @.code + 1

end

Collations are definitely the issue, since the orginal query works as intended (returning all long_value columns with non-ASCII characters) when reformulated with an explicit collation: select * from Item_Detail

where (long_value collate Latin1_General_BIN2) like (N'%[' + nchar(128) + N'-' + nchar(65535) + N']%')

|||

Kevin Rodgers wrote:

Kevin Rodgers wrote:

Here is a diagnostic query that suggests to me that the problem has something to do with collations (a topic I had been blissfully ignorant of). If the server's default collation or the database's collation is Latin1_General_BIN2, it returns the expected result; but if the collation is SQL_Latin1_General_CP1_CI_AS, it returns 21,230 Unicode characters between 0 and 65535 inclusive which are equal to nchar(65535). Most disturbing is the fact that 0 (NULL) and 32 (SPACE) are among them, which might explain my original problem with the character range. Try this out on different servers and databases:

declare @.code as integer

set @.code = 0

while (@.code < 65536)

begin

if (nchar(@.code) = nchar(65535))

print cast(@.code as nchar) + ' equal'

set @.code = @.code + 1

end

Collations are definitely the issue, since the orginal query works as intended (returning all long_value columns with non-ASCII characters) when reformulated with an explicit collation: select * from Item_Detail

where (long_value collate Latin1_General_BIN2) like (N'%[' + nchar(128) + N'-' + nchar(65535) + N']%')

Unfortunately, the Latin1_General_CI_AS collation does not work like the Latin1_General_BIN2 collation. Can someone explain that?

comparing nvarchar(max) column using like to non-ASCII range

Our database defines the long_value column as nvarchar(max). I want to find out which rows actually contain non-ASCII characters in that column, but this clause also returns rows with only ASCII characters:

where long_value like (N'%[' + nchar(128) + N'-' + nchar(65535) + N']%')

What am I doing wrong?

It seems to be the fact that nchar(65535) has no length. I couldn't find any really good reference to cover this (perhaps this thread: http://groups.google.com/group/comp.lang.javascript/tree/browse_frm/month/2004-09/7d3603a75c1550f3?rnum=91&_done=%2Fgroup%2Fcomp.lang.javascript%2Fbrowse_frm%2Fmonth%2F2004-09%3F), but if you run these statements:

select N'%[' + nchar(128) + N'-' + nchar(65535) + N']%'

select N'%[' + nchar(128) + N'-' + nchar(65534) + N']%'

You will see that there is no second character after the - in the second string. This should do what you want:

declare @.long_value nvarchar(max)
set @.long_value = N'abcdefghijklmnopqrstuvwxyz' + nchar(129)
select case when @.long_value like (N'%[' + nchar(128) + N'-' + nchar(65534) + N']%') then 'yes' else 'no' end

and you could add a check for the nchar(65535) also...

|||

Louis Davidson wrote:

It seems to be the fact that nchar(65535) has no length. I couldn't find any really good reference to cover this (perhaps this thread: http://groups.google.com/group/comp.lang.javascript/tree/browse_frm/month/2004-09/7d3603a75c1550f3?rnum=91&_done=%2Fgroup%2Fcomp.lang.javascript%2Fbrowse_frm%2Fmonth%2F2004-09%3F), but if you run these statements:

select N'%[' + nchar(128) + N'-' + nchar(65535) + N']%'

select N'%[' + nchar(128) + N'-' + nchar(65534) + N']%'

You will see that there is no second character after the - in the second string.

I can't corroborate your results. I'm runing SQL Server 2005, both an unpatched Enterprise Edition and the SP1 + build 2153 hotfix Developer Edition. Connected to either server's master database, the result of both queries is displayed in Management Studio with an empty box on either side of the dash. Furthermore, the results of the following queries are 1, 2, 3, and 7 as expected:

selectlen(nchar(65535));

selectlen(nchar(128)+nchar(65535));

selectlen(nchar(128)+ N'-'+nchar(65535));

selectlen(N'%['+nchar(128)+ N'-'+nchar(65535)+ N']%');

Louis Davidson wrote:

This should do what you want:

declare @.long_value nvarchar(max)
set @.long_value = N'abcdefghijklmnopqrstuvwxyz' + nchar(129)
select case when @.long_value like (N'%[' + nchar(128) + N'-' + nchar(65534) + N']%') then 'yes' else 'no' end

and you could add a check for the nchar(65535) also...

Well, the obvious way to add a check for nchar(65535) is to include it in the range:

N'%['+nchar(128)+ N'-'+nchar(65534)+nchar(65535)+ N']%'

Which, if it worked, would indicate a bug in character range matching. But it fails in the same way as the original.

Here is a diagnostic query that suggests to me that the problem has something to do with collations (a topic I had been blissfully ignorant of). If the server's default collation or the database's collation is Latin1_General_BIN2, it returns the expected result; but if the collation is SQL_Latin1_General_CP1_CI_AS, it returns 21,230 Unicode characters between 0 and 65535 inclusive which are equal to nchar(65535). Most disturbing is the fact that 0 (NULL) and 32 (SPACE) are among them, which might explain my original problem with the character range. Try this out on different servers and databases:

declare @.code as integer

set @.code = 0

while(@.code < 65536)

begin

if(nchar(@.code)=nchar(65535))

printcast(@.code asnchar)+' equal'

set @.code = @.code + 1

end

|||

Kevin Rodgers wrote:

Here is a diagnostic query that suggests to me that the problem has something to do with collations (a topic I had been blissfully ignorant of). If the server's default collation or the database's collation is Latin1_General_BIN2, it returns the expected result; but if the collation is SQL_Latin1_General_CP1_CI_AS, it returns 21,230 Unicode characters between 0 and 65535 inclusive which are equal to nchar(65535). Most disturbing is the fact that 0 (NULL) and 32 (SPACE) are among them, which might explain my original problem with the character range. Try this out on different servers and databases:

declare @.code as integer set @.code = 0 while (@.code < 65536) begin if (nchar(@.code) = nchar(65535)) print cast(@.code as nchar) + ' equal' set @.code = @.code + 1 end

Collations are definitely the issue, since the orginal query works as intended (returning all long_value columns with non-ASCII characters) when reformulated with an explicit collation: select * from Item_Detail where (long_value collate Latin1_General_BIN2) like (N'%[' + nchar(128) + N'-' + nchar(65535) + N']%')|||

Kevin Rodgers wrote:

Kevin Rodgers wrote:

Here is a diagnostic query that suggests to me that the problem has something to do with collations (a topic I had been blissfully ignorant of). If the server's default collation or the database's collation is Latin1_General_BIN2, it returns the expected result; but if the collation is SQL_Latin1_General_CP1_CI_AS, it returns 21,230 Unicode characters between 0 and 65535 inclusive which are equal to nchar(65535). Most disturbing is the fact that 0 (NULL) and 32 (SPACE) are among them, which might explain my original problem with the character range. Try this out on different servers and databases:

declare @.code as integer set @.code = 0 while (@.code < 65536) begin if (nchar(@.code) = nchar(65535)) print cast(@.code as nchar) + ' equal' set @.code = @.code + 1 end

Collations are definitely the issue, since the orginal query works as intended (returning all long_value columns with non-ASCII characters) when reformulated with an explicit collation: select * from Item_Detail where (long_value collate Latin1_General_BIN2) like (N'%[' + nchar(128) + N'-' + nchar(65535) + N']%')

Unfortunately, the Latin1_General_CI_AS collation does not work like the Latin1_General_BIN2 collation. Can someone explain that?

Comparing Nulls

Hi All,
I am in the middle of checking data between SQL servers.
I am not getting any rows returned because in one server the amount is null
in the other is is a number. why aren't these unequal amounts coming in?
I have the same issue with date fileds.
my code is:
select a.accountkey,b.accountkey as BRacct,a.ContractID,a.ContractAmount,
b.ContractAmount as BRconAmt
from ACrepl..maincontracts a inner join BR11.AC.dbo.tblmaincontracts b
on a.contractId = b.contractID
where a.contractAmount <> b.contractAmount
TIA,
JoeThe only comparison allowed, when ansi_nulls is on, for null values are [not]
is null. Null can not be compared even with itself.
Try:
select
a.accountkey,
b.accountkey as BRacct,
a.ContractID,
a.ContractAmount,
b.ContractAmount as BRconAmt
from
ACrepl..maincontracts a
inner join
BR11.AC.dbo.tblmaincontracts b
on a.contractId = b.contractID
where
a.contractAmount <> b.contractAmount
or (contractAmount + b.contractAmount) is null
go
AMB
"jaylou" wrote:

> Hi All,
> I am in the middle of checking data between SQL servers.
> I am not getting any rows returned because in one server the amount is nul
l
> in the other is is a number. why aren't these unequal amounts coming in?
> I have the same issue with date fileds.
> my code is:
> select a.accountkey,b.accountkey as BRacct,a.ContractID,a.ContractAmount,
> b.ContractAmount as BRconAmt
> from ACrepl..maincontracts a inner join BR11.AC.dbo.tblmaincontracts b
> on a.contractId = b.contractID
> where a.contractAmount <> b.contractAmount
> TIA,
> Joesqlsql

Comparing Keys

Hi! I Have a Table with composed Primary Key and need to validate in a
stored procedure that new rows from a temporary table that would be inserted does not exist already.
I use to have a code similar to this for many tables, even with composed keys

Delete Det_Order

Where Ltrim(Rtrim(Convert(Char(20),id))) +
Ltrim(Rtrim(Convert(Char(20),renglon)))

In (Select Ltrim(Rtrim(Convert(Char(20),id))) +
Ltrim(Rtrim(Convert(Char(20),renglon)))
From ##TmpDet_Order )


But I just dont now what happen with this case that the process just hold and never ends. Both fields that I have in as keys are integer. Would you have any idea of what happens? or maybe you can tell me another way to make the same. Please ...

I have also tried this..
Delete from Det_Order
Where [id] + renglon

In (Select [id] + renglon
From ##Det_Order )

Rather than use the IN condition, try the TSQL DELETE extension; something like:

Delete Det_Order
from ##TmpDet_Order a
join Det_Order b
on a.[id] = b.[id]
and a.renglon = b.renglon

In general, I try to use the TSQL extensions sparingly, but this seems like a good spot for use.

|||I have also tried Kent, but have the same problem |||

Try running this query and let us know record count returned; have you tried limiting the number of rows being deleted in a single batch?

select count(*)
from ##TmpDet_Order (nolock) a
join Det_Order (nolock) b
on a.[id] = b.[id]
and a.renglon = b.renglon

|||

ok, my rowcount is 4677

The purpose of this process is to run every day and getting data from a day before.

|||

Well, that isn't nominally enough to grieve anything. My knee-jerk reaction is to run profiler to see what is going on; you also might be able to get at this by running SP_LOCK while your delete query is hung because it sounds like you might be blocked. If you will perform a search on this site you will find posts that discuss diagnosis of performance problems.

More importantly, I'd like to recruit opinions from others that are more experienced.

Kent

|||I could be a resource contention issue. I would also check the indexing on the key.|||

How long does it take for a similar SELECT query to execute?

select

b.*

from ##TmpDet_Order a
join Det_Order b
on a.[id] = b.[id]
and a.renglon = b.renglon

I have occasionally encountered problems with DELETE statements that are deleting from large tables.

How many rows are in Det_Order?

How many indexes are associated with Det_Order?

Are any of the indexes on Det_Order CLUSTERED indexes? Maintaining clustered indexes has a performance penalty -- but I would not have expected it to be substantial when deleting only 4700 rows.

|||

Ok, I have a clusterd index on the field last_date_change and have anotherone with id + renglon.

Could it afect the order?

|||

If your table is large, my understanding is that considerable effort must be expended to adjust the data to conform to the clustered index. Such maintenance is required merely as a result of deleting records -- it does not depend on the columns you used to determine which records to delete.

But you cannot have TWO clustered indexes on a single table.

You never told us how large your table is (rows, as well as megabytes), only that you were deleting 4700 rows, or so.

Dan

|||Oh yes, my table has 5,917,688 rows.. and it has a cluster index by the field of date.|||

That sounds like a lot of data to move about, to maintain the clustered index.

In any case, I would like to suggest that you make a copy of your table and change the CLUSTERED index to a NON-CLUSTERED index. Test your query against this copy of the table. (Unfortunately this change will increase the size of your indexes (by approx. 50 MB, if a DATETIME column is the only column of the CLUSTERED index), but I am expecting that it will improve the performance of any INSERT, UPDATE, and DELETE actions you perform on this table.)

If you frequently perform millions of INSERT, UPDATE, and DELETE actions on this table, you may want to consider using FILLFACTOR = 50, or maybe 80, depending on the quantity of such actions. (For some of my processing, where I have only 100,000 rows in a table when I create the index, but at later stages of processing I add another 200,000 rows that will be intermingled with these others, I set the FILLFACTOR = 30 to leave room in the index for the new rows. Some of my INSERT actions that formerly took 7-10 minutes then took only a few seconds.)

|||Thanks everyobody : )

Thursday, March 22, 2012

Comparing Dates in Previous and Current Rows

Dear All,
I am having textboxes on two detail rows (as an example), I want to calculate the date difference between two textbox date values. The date difference is between the previous row and the current row. As an example:

A B
5/2/2005 09:39:02 AM5/3/2005 06:29:32 PM
5/4/2005 08:21:31 AM5/5/2005 07:02:29 PM
5/6/2005 09:52:33 AM5/6/2005 07:04:31 PM
5/7/2005 09:20:33 AM5/8/2005 08:26:36 AM

I want to do A2-B1 in the report. I need to calculate the date difference between 5/4/2005 8:21:31 AM and 5/3/2005 6:29:32 PM as an example. The same thing for A3 and B2. ...
Do you have any idea how I can achieve this? Is there a way to do that? Do I need some special code to accomplish this?
Thank you for your help.
BTHi,
a way is to create a store proc that make the calculation
(fetching results in a temp table).|||
yes you can
you can use formula like this
=ReportItems!A.Value.Subtract(ReportItems!B.Value)
where A and B are the name of the cells in your table
noteSubtractfunction in datetime returns timespan|||

Hi,
IMHO the formula
ReportItems!A.Value.Subtract(ReportItems!B.Value)
calculate difference between value of the same row.
I've understood that Salmiya have to calculate between a value of column A with
value of column B, but of previous row.

|||Yes you are right. I have to compare previous row with current one.
Thank you.|||tblDates :
PK A B
----------------
1 5/2/2005 09:39:02 AM 5/3/2005 06:29:32 PM
2 5/4/2005 08:21:31 AM 5/5/2005 07:02:29 PM
3 5/6/2005 09:52:33 AM 5/6/2005 07:04:31 PM
4 5/7/2005 09:20:33 AM 5/8/2005 08:26:36 AM

You can do :
Select [Difference] = A-IsNull((Select B from tblDates where PK=2 ),0) from tblDates where PK=3
HTH,
Best Regards,
Hemchand|||Hi.....
U can use =ReportItems!textbox1.Value - ReportItems!textbox2.Value
Then U will get it..........
Best Regards....