Showing posts with label example. Show all posts
Showing posts with label example. Show all posts

Friday, March 30, 2012

Prediction Query in MS Association Rules

Hi!

I'm building a mining model wiht MS Association Rules. After processing this model, the result includes some rules(example):

E = Existing, C = Existing -> B = Existing
F = Existing -> E = Existing
C = Existing, B = Existing -> E = Existing
F = Existing -> B = Existing
B = Existing, A = Existing -> C = Existing
F = Existing, B = Existing -> E = Existing
F = Existing, E = Existing -> B = Existing
D = Existing -> A = Existing
C = Existing -> A = Existing
E = Existing, A = Existing -> B = Existing

I want to buid a query that has two or more items on the left of the rules, example: E = Existing, C = Existing -> B = Existing
->I want to buid a query to predict that: when a customer buy 'E' and 'C' then he likely buys 'B'


All the rules are used when you use AR for prediction. The first place to look is the prediction query builder. There is a button on the top to switch the mode from batch to singleton. With a singleton prediction you can manually specify the inputs for your query.

The prediction function you need to specify is something like "Predict(<my nested table name>, 5)". To build such a prediction in the query builder, select Prediction Function, then Predict, then type the name of your nested table, comma, then the number of recommendations you want into the parameters box.

To see the query select the SQL mode from the toolbar.

Let me know if this helps or if you were looking for some other type of answer

THanks

-Jamie

|||

Hi!

Thanks for interesting in my question!

My domain has two tables: Customer (Customer_ID, Name, ....) and Purchase (Customer_ID, Product_Name, Quantity,...)

Creating Mining Model:

Create Mining Model ProductPredict{

Customer_ID long key,

Purchase Table Predict {Product_Name text key}

}

So, when i buid a query such as:

Select Predict(Purchase, 3)

From ProductPredict Prediction Join

(Select 'A' As Product_Name

) as customer

On [ProductPredict].[Purchase].Product_Name = customer.Product_Name;

Result as all item in the right side of the rules contain 'A' in the left side.

But I want to buid a query that result as all item in the right side of the rules contain 'A' and 'B' in the left side.?

Summary: I want to buid a query that result as all item in the right side of the rules contain some items in the left side?

|||

You need a query such as

Select Predict(Purchase, 3)

From ProductPredict Prediction Join

(select

(Select 'A' As Product_Name UNION Select 'B' AS Product_Name)

as Products

) as customer

On [ProductPredict].[Purchase].Product_Name = customer.Product_Name;

This will cause rules with A and B to fire. You may still get predictions based on A alone and B alone, though, depending on their probability and lift.

|||

Hi!

Thank you very much! That's interesting, but when i run that query, it has error, so the correct query is:

Select PredictAssociation(Purchase, 3)

From ProductPredict Prediction Join

(

select

( Select 'A' As Product_Name

UNION

Select 'B' AS Product_Name

)as Products

) as customer

On [ProductPredict].[Purchase].Product_Name = [customer].[Products].Product_Name

|||Predict is a polymorphic function - DMX maps it to the appropriate function based on the model that's being queried. In this case, it maps to PredictAssociation so you should get the same results either way. What errors did you see with the earlier query?

Predict Probability in Decision Trees

Hello,

I installed the bike buyer example and i am learning the DMX language. Now i wrote the following query (using MS decision trees):

SELECT
T.[Last Name],
[Bike Buyer],
PredictProbability(Predict([Bike Buyer])) AS [Probability]
From
[v Target Mail]
PREDICTION JOIN
OPENQUERY
(....... And so on..)

Now the result is surprising to me. In the resulttabel all the probabilities are equal.

Bike Buyer Probability
1 0.99994590500919611
0 0.99994590500919611
0 0.99994590500919611
0 0.99994590500919611
0 0.99994590500919611
1 0.99994590500919611

and so on.

Now i am wondering what predictProbability means. I thought that PredictProbability meant the probability that the prediction is correct. Now all the probabilities are the same and the input is different. Can somebody tell me what PredictProbability means or am I using it wrong?

Thanx in advance,

Joris Valkonet

This is an interesting query - I would write it as "PredictProbability([Bike Buyer])" however, the syntax is semantically the same. Another thing to try is PredictNodeId([Bike Buyer]) to see exactly the node used for the prediction and then check that node to see the distribution.

It's possible that PredictProbability(Predict([Bike Buyer])) is exposing a bug and you should use the simpler syntax above.

HTH

-Jamie

|||

Thank you for your reply.
I modified my query to PredictProbability([bike buyer]) and this made no difference. The result is the same.

Then I checked the PredictNodeId([Bike Buyer]) and this is the result:

FirstName

LastName

bike buyer

Expression

Abby

Malhotra

1

000000003

Abby

Prasad

0

000000003

Abby

Rodriguez

0

000000003

Abby

Srini

0

000000003

Abigail

Brown

0

000000003

Abigail

Bryant

1

000000003

Abigail

Davis

0

000000003

Abigail

Flores

0

000000003

Now the nodeId is the rootnode of the tree and the probability of the root node is indeed the value 0.99994590500919611. So if I understand this correct, the rootnode of the decision tree is used for all probability predictions. Is this correct?

(Maybe my query is incorrect and therefore the whole DMX query below)

SELECT
t.[FirstName],
t.[LastName],
[bike buyer],
PredictNodeId([bike buyer]) AS [Predicted NodeId]
From
[v Target Mail]
PREDICTION JOIN
OPENQUERY([Adventure Works DW],
'SELECT
[Gender],
[FirstName],
[MiddleName],
[LastName],
[BirthDate],
[MaritalStatus],
[EmailAddress],
[YearlyIncome],
[TotalChildren],
[NumberChildrenAtHome],
[HouseOwnerFlag],
[NumberCarsOwned],
[AddressLine1],
[AddressLine2],
[Phone]
FROM
[dbo].[ProspectiveBuyer]
') AS t
ON
[v Target Mail].[First Name] = t.[FirstName] AND
[v Target Mail].[Middle Name] = t.[MiddleName] AND
[v Target Mail].[Last Name] = t.[LastName] AND
[v Target Mail].[Birth Date] = t.[BirthDate] AND
[v Target Mail].[Marital Status] = t.[MaritalStatus] AND
[v Target Mail].[Gender] = t.[Gender] AND
[v Target Mail].[Email Address] = t.[EmailAddress] AND
[v Target Mail].[Yearly Income] = t.[YearlyIncome] AND
[v Target Mail].[Total Children] = t.[TotalChildren] AND
[v Target Mail].[Number Children At Home] = t.[NumberChildrenAtHome] AND
[v Target Mail].[House Owner Flag] = t.[HouseOwnerFlag] AND
[v Target Mail].[Number Cars Owned] = t.[NumberCarsOwned] AND
[v Target Mail].[Address Line1] = t.[AddressLine1] AND
[v Target Mail].[Address Line2] = t.[AddressLine2] AND
[v Target Mail].[Phone] = t.[Phone]

|||

Something wierd is happening here - Predict([Bike Buyer]), which is equivalent to saying just [Bike Buyer], returns the highest probability value for the prediction. That means that whatever node the tree goes to, it will return the highest probability state for that node. I can't see how you are getting different prediction results from the same node.

However, something else struck me as odd. The predict probability of 0.999... doesn't seem right. In fact the only time I have ever seen anything like this is when all the data at the node was missing, and the actual non-missing values only recieved the bayesian prior. By default prediction never returns missing (if you are asking for a value, it assumes you want an actual value). If you do a Predict([Bike Buyer], INCLUDE_MISSING) you will get predictions that include the missing value. If there were no observations in the training data of the states 1 and 0 then they would only have their prior probabilities which would be equal and then the choice of a state would be arbitrary.

What confuses me is that PredictProbabililty should behave the same way.

Can you do a SELECT FLATTENED NODE_DISTRIBUTION FROM [v Target Mail].CONTENT and post the results?

Thanks

|||

Thanks for your analysis. I tried to post the whole result of the query "SELECT FLATTENED NODE_DISTRIBUTION FROM [v Target Mail].CONTENT ", but then an unknown error occurs. I think because the resulttable is to big. So... i will give the first top rows of the result.

Thanks....

ATTRIBUTE_NAME

ATTRIBUTE_VALUE

SUPPORT

PROBABILITY

VARIANCE

.VALUETYPE

Bike Buyer

Missing

0

0.000432432

0

1

Bike Buyer

0.494048907

18484

0.999567568

0.249964584

3

Birth Date

5.45E-06

0

0

16892175

7

Birth Date

0.080783333

0

0

0

8

Birth Date

22673.85945

0

0

16892175

9

Date First Purchase

-0.000941648

0

0

76769.40019

7

Date First Purchase

3242.190525

0

0

0

8

Date First Purchase

37852.25763

0

0

76769.40019

9

Number Cars Owned

-0.079859089

0

0

1.295870198

7

Number Cars Owned

301.5988757

0

0

0

8

Number Cars Owned

1.502705042

0

0

1.295870198

9

Number Children At Home

-0.015796323

0

0

2.318367003

7

Number Children At Home

9.530872281

0

0

0

8

Number Children At Home

1.004057563

0

0

2.318367003

9

Total Children

-0.003992138

0

0

2.599718694

7

Total Children

2.60E+01

0

0

0

8

Total Children

1.844351872

0

0

2.599718694

9

Yearly Income

2.02E-06

0

0

1042319181

7

Yearly Income

116.9361238

0

0

0

8

Yearly Income

57305.77797

0

0

1042319181

9

36.04143269

0

0

0.166743652

11

Bike Buyer

Missing

0

0.000205761

0.00E+00

1

Bike Buyer

1

4858

0.999794239

0

3

1

0

0

5.14E-07

11

Bike Buyer

Missing

0

0.000536481

0

1

Bike Buyer

0.507518797

1862

0.999463519

0.249943468

3

Date First Purchase

-0.010374704

0

0

1160.841795

7

Date First Purchase

679.5475673

0

0

0

8

Date First Purchase

37822.50644

0

0

1160.841795

9

Geography Key

0.000170418

0

0

36083.71548

7

Geography Key

1.990727272

0

0

0

8

Geography Key

236.4581096

0

0

36083.71548

9

Number Cars Owned

-0.072103841

0

0

1.254812169

7

Number Cars Owned

2.37E+01

0

0

0

8

Number Cars Owned

1.523630505

0

0

1.254812169

9

Yearly Income

1.29E-06

0

0

1131929217

7

Yearly Income

7.102048444

0

0

0

8

Yearly Income

60633.72718

0

0

1131929217

9

392.8960717

0

0

0.113511443

11

Bike Buyer

Missing

0

0.000123274

0

1

Bike Buyer

0.256103576

8110

0.999876726

0.190514534

3

|||

I see the problem - you have Bike Buyer modeled as continuous where it should be discrete. PredictProbability was giving you the probability that BikeBuyer exists in this case. To get the confidence of the prediction for continuous predictions, you want to use PredictStdev.

Essentially what was happening was that the model was creating a linear regression to predict bike buyer rather than a classification model. Changing the content type to "Discrete" will fix your issue. If you are creating the model from the source table (rather than a cube), you can click on "Detect" on the data types page of the wizard and it will automatically determing that Bike Buyer should be discrete.

Wednesday, March 28, 2012

precision

Precision is the number of digits in a number. Scale is the number of digits
to the right of the decimal point in a number. For example, the number
123.45 has a precision of 5 and a scale of 2.
Is the above statement true? or Is the precision 3 and the scale is 2?
Thank you in advance.In your example 123.45 the precision (p) is 5 and scale (s) is 2.
For more information:
http://msdn.microsoft.com/library/en-us/tsqlref/ts_de-dz_3grn.asp
--
Rohtash Kapoor
http://www.sqlmantra.com
"Praetorian Guard" <praetorian@.gatekeeper.com> wrote in message
news:O7cosLP1DHA.832@.TK2MSFTNGP09.phx.gbl...
> Precision is the number of digits in a number. Scale is the number of
digits
> to the right of the decimal point in a number. For example, the number
> 123.45 has a precision of 5 and a scale of 2.
> Is the above statement true? or Is the precision 3 and the scale is 2?
> Thank you in advance.
>|||So on this one 1234567890.1234 the precision is 14 and the scale is 4 same
as (14,4)?
"Rohtash Kapoor" <rohtash_nospam@.sqlmantra.com> wrote in message
news:%230BQSWP1DHA.540@.tk2msftngp13.phx.gbl...
> In your example 123.45 the precision (p) is 5 and scale (s) is 2.
> For more information:
> http://msdn.microsoft.com/library/en-us/tsqlref/ts_de-dz_3grn.asp
> --
> Rohtash Kapoor
> http://www.sqlmantra.com
> "Praetorian Guard" <praetorian@.gatekeeper.com> wrote in message
> news:O7cosLP1DHA.832@.TK2MSFTNGP09.phx.gbl...
> > Precision is the number of digits in a number. Scale is the number of
> digits
> > to the right of the decimal point in a number. For example, the number
> > 123.45 has a precision of 5 and a scale of 2.
> >
> > Is the above statement true? or Is the precision 3 and the scale is 2?
> >
> > Thank you in advance.
> >
> >
>|||Yes
>--Original Message--
>So on this one 1234567890.1234 the precision is 14 and
the scale is 4 same
>as (14,4)?
>"Rohtash Kapoor" <rohtash_nospam@.sqlmantra.com> wrote in
message
>news:%230BQSWP1DHA.540@.tk2msftngp13.phx.gbl...
>> In your example 123.45 the precision (p) is 5 and scale
(s) is 2.
>> For more information:
>> http://msdn.microsoft.com/library/en-us/tsqlref/ts_de-
dz_3grn.asp
>> --
>> Rohtash Kapoor
>> http://www.sqlmantra.com
>> "Praetorian Guard" <praetorian@.gatekeeper.com> wrote in
message
>> news:O7cosLP1DHA.832@.TK2MSFTNGP09.phx.gbl...
>> > Precision is the number of digits in a number. Scale
is the number of
>> digits
>> > to the right of the decimal point in a number. For
example, the number
>> > 123.45 has a precision of 5 and a scale of 2.
>> >
>> > Is the above statement true? or Is the precision 3
and the scale is 2?
>> >
>> > Thank you in advance.
>> >
>> >
>>
>
>.
>

Precedence of MAX and WHERE

Hi
I'd like to create a query which returns the MAX of a group of dates so long
as the number is less than a given date. For example :
SELECT MAX(date), username
FROM mydatatable
WHERE date < '01/01/2005'
GROUP BY username
Will this do what I expect and return the username and date which is the
most recent before 01/01/2005 ?
Thanks
AndrewHi,
Your query looks good.
Thanks
Hari
SQL Server MVP
"Andrew Webb" <andrew.webb@.eme-med.co.uk> wrote in message
news:OTlNYqfuFHA.1572@.TK2MSFTNGP10.phx.gbl...
> Hi
> I'd like to create a query which returns the MAX of a group of dates so
> long as the number is less than a given date. For example :
>
> SELECT MAX(date), username
> FROM mydatatable
> WHERE date < '01/01/2005'
> GROUP BY username
>
> Will this do what I expect and return the username and date which is the
> most recent before 01/01/2005 ?
> Thanks
> Andrew
>|||You could compare these queries and see which one yields the results you
want. Word problems are tough to solve, usually better to provide specs as
described in http://www.aspfaq.com/5006 . Also, "date" is a really bad name
for a column. Not only is it a reserved word, it is also very tough to
decipher it... date of WHAT? Finally, do not use m/d/y or d/m/y date
formats when hard-coding date strings. The safest approach here is to use
YYYYMMDD format, then this can't be by software or humans.
CREATE TABLE dbo.myDataTable
(
username VARCHAR(32),
eventDate SMALLDATETIME
)
GO
SET NOCOUNT ON
INSERT myDataTable SELECT 'bob','20040101'
INSERT myDataTable SELECT 'bob','20050201'
INSERT myDataTable SELECT 'frank','20040101'
INSERT myDataTable SELECT 'frank','20040725'
GO
SELECT username, MAX(eventDate)
FROM dbo.myDataTable
WHERE eventDate < '20050101'
GROUP BY username
SELECT username, MAX(eventDate)
FROM dbo.myDataTable
GROUP BY username
HAVING MAX(eventDate) < '20050101'
GO
DROP TABLE dbo.myDataTable
GO
"Andrew Webb" <andrew.webb@.eme-med.co.uk> wrote in message
news:OTlNYqfuFHA.1572@.TK2MSFTNGP10.phx.gbl...
> Hi
> I'd like to create a query which returns the MAX of a group of dates so
> long as the number is less than a given date. For example :
>
> SELECT MAX(date), username
> FROM mydatatable
> WHERE date < '01/01/2005'
> GROUP BY username
>
> Will this do what I expect and return the username and date which is the
> most recent before 01/01/2005 ?
> Thanks
> Andrew
>|||Andrew,

> Will this do what I expect and return the username and date which is the
> most recent before 01/01/2005 ?
It is correct, but it could be more than one. It will select each username
and the max date for those username with date values less than '20050101'. I
f
a username does not have date values in this range then it will not appear i
n
the result.
AMB
"Andrew Webb" wrote:

> Hi
> I'd like to create a query which returns the MAX of a group of dates so lo
ng
> as the number is less than a given date. For example :
>
> SELECT MAX(date), username
> FROM mydatatable
> WHERE date < '01/01/2005'
> GROUP BY username
>
> Will this do what I expect and return the username and date which is the
> most recent before 01/01/2005 ?
> Thanks
> Andrew
>
>

Friday, March 9, 2012

possible to free-text search two tables at once in one query?

is it possible to use a full-text search query on two tables at the same time? for example, say i have a table of items, and another table of descriptions... would i be able to search both those tables for a string in one query? they are currently in indexed in separate catalogs, would i need to put them both in one?

thanksUse subqueries, maybe even with UNION [ALL] if the result set will contain the same number of fields in the same order, and of the same data types.

Wednesday, March 7, 2012

Possible to add columns to the right-hand side of a matrix?

Is it possible to add columns to the right-hand side of a matrix? Does anyone
have an example of joining a matrix and a table? Our report designs are never
as simple as the AdventureWorks examples. Currently, we use Actuate, which
allows for practically unlimited possibilities for report layout. We are
investigating migrating to Reporting Services, and we would appreciate any
tips/examples for achieving more complex layouts. Thanks!Not sure that I understand your question.
The Idea with the matrix is to dynammically add columns based on the group
you picked for the columns.
Maybe introduce some black"null" members in that group.
i.e if the dataset read something like
rowgroup1,columngroup1,measurefield1
a x1 10
b x2 20
c x3 30
d x4 40
try make it look like
rowgroup1,columngroup1,measurefield1
a x1 10
b x2 20
c x3 30
d x4 40
a Z NULL
b Z1 NULL
then on the report in the measure field ...iif (columngroup1='Z',"",.....)
marcell
We moved to ReportingServices with schemtisism, and have been waiting to
reach a stage where we hit a wall.
Still hasnt happened and I have developed some insanely complicated reports.
Flexibility is great!
"chatlineboy" <chatlineboy@.discussions.microsoft.com> wrote in message
news:79300D9D-0EFA-4047-B8B5-A231FBD68115@.microsoft.com...
> Is it possible to add columns to the right-hand side of a matrix? Does
> anyone
> have an example of joining a matrix and a table? Our report designs are
> never
> as simple as the AdventureWorks examples. Currently, we use Actuate, which
> allows for practically unlimited possibilities for report layout. We are
> investigating migrating to Reporting Services, and we would appreciate any
> tips/examples for achieving more complex layouts. Thanks!
>|||Thanks brickmouse! Yes, we had thought of that, also just introducing extra
groups with repeating values. We are continuing to evaluate Reporting
Services, but so far we are not convinced it is as flexible as Actuate.
"brickmouse" wrote:
> Not sure that I understand your question.
> The Idea with the matrix is to dynammically add columns based on the group
> you picked for the columns.
> Maybe introduce some black"null" members in that group.
> i.e if the dataset read something like
> rowgroup1,columngroup1,measurefield1
> a x1 10
> b x2 20
> c x3 30
> d x4 40
> try make it look like
> rowgroup1,columngroup1,measurefield1
> a x1 10
> b x2 20
> c x3 30
> d x4 40
> a Z NULL
> b Z1 NULL
> then on the report in the measure field ...iif (columngroup1='Z',"",.....)
>
> marcell
>
> We moved to ReportingServices with schemtisism, and have been waiting to
> reach a stage where we hit a wall.
> Still hasnt happened and I have developed some insanely complicated reports.
> Flexibility is great!
>
> "chatlineboy" <chatlineboy@.discussions.microsoft.com> wrote in message
> news:79300D9D-0EFA-4047-B8B5-A231FBD68115@.microsoft.com...
> > Is it possible to add columns to the right-hand side of a matrix? Does
> > anyone
> > have an example of joining a matrix and a table? Our report designs are
> > never
> > as simple as the AdventureWorks examples. Currently, we use Actuate, which
> > allows for practically unlimited possibilities for report layout. We are
> > investigating migrating to Reporting Services, and we would appreciate any
> > tips/examples for achieving more complex layouts. Thanks!
> >
>
>