Data Mining in Finance (DMF)
Spring 1999
Andreas S. Weigend, Stern School of Business
About this course
Data Mining in Finance (DMF) develops links between Information Systems, Statistics and Finance. It focuses on current problems in finance such as understanding and managing risk, and building and evaluating trading models. It covers the foundations of modern modeling, knowledge discovery and data mining techniques as well as specific methods including neural networks and model-based clustering. It highlights the assumptions of different model classes, stresses the critical evaluation and comparison with established methods, and offers techniques for interpreting the results. Weekly computer assignments are based on state-of-the-art software. Possibilities of in-depth group projects in conjunction with major financial firms exist. Towards the end of the semester, several Wall Street practitioners present their perspectives on data mining in finance.

In addition to calculus and probability and the IS, Finance and Statistics core courses, DMF builds on the following courses:
Statistical Inference and Regression Analysis (B90.3302) (DMF heavily relies on the second half of B90.3302 ),
Regression and Multivariate Data Analysis (B90.2301 = B90.3311 for PhDs ), and
Forecasting Time Series Data (B90.2302 = B90.3312 for PhDs = C22.0018 for undergraduates) or another time series course.

Deeper insights can be gained if DMF is taken after or at the same time with:
Bayesian Inference and Statistical Decision Theory (B90.3305), and
Stochastic Processes I (B90.3321).

Comparing DMF to related courses in the school, DMF is statistically more advanced than Knowledge Systems in Organizations (B20.3336 = C20.3336), and  also more theoretical and broad than Risk Management Systems (B20.3351).

A more detailed general description includes the teaching philosophy.
Furthermore, the book chapter Data Mining in Finance: Report from the Post-NNCM-96 Workshop on Teaching Computer Intensive Methods for Financial Modeling and Data Analysis (as pdf, as ps) provides some history of this course.

 
Logistics
The logistics page contains the general information about the instructor and teaching assistant, readings, software, and the grading.

If you are looking for specific files, the following links will take you directly to the directories:
Notes
Readings
Homeworks
MatlabDMF (written for this course)
Software (other software)

 
Schedule (session-by-session)

#

Date

 Day Topic
1 1/20/99 Wed Data Mining and Data Snooping
2 1/25/99 Mon Learning from Data: Bayes, 7 Steps
3 1/27/99 Wed Bootstrap
4 2/1/99 Mon Evaluation and Model Risk
5 2/3/99 Wed Data and Representation, Linear Models
6 2/8/99 Mon Nonlinear Models (see also the Information Theorynotes.
7 2/10/99 Wed Neural Networks
PD 2/15/99 Mon (no class)
8 2/17/99 Wed Neuro-fuzzy Models
9 2/22/99 Mon Conditional Normal Predictions
10 2/24/99 Wed Conditional Non-Normal Predictions
11 3/1/99 Mon Tail Predictions
12 3/3/99 Wed Computing Non-normal Implied Densities
13 3/8/99 Mon Principal Component Analysis
14 3/10/99 Wed Independent Component Analysis
SB 3/15/99 Mon (no class)
SB 3/17/99 Wed (no class)
The schedule after spring break is likely to shift.
There will be one or two additional guest speakers
15 3/22/99 Mon  
16 3/24/99 Wed  
17 3/29/99 Mon (Martin, Robust Approaches)
18 3/31/99 Wed Markov Models
19 4/5/99 Mon Hidden Markov Models
20 4/7/99 Wed (Dhar, How Investors Look at Models)
WS 4/10/99 Sat (Zimmermann, Portfolio Workshop)
21 4/12/99 Mon Classification
22 4/14/99 Wed (Grody, Practical Implications of Implementing Enterprise Wide Risk Management Systems)
23 4/19/99 Mon Style Analysis: Traders (Clustering)
24 4/21/99 Wed Style Analysis: Mutual Funds
25 4/26/99 Mon Style Analysis: Hedge Funds
26 4/28/99 Wed (Li, Risk in Practice)
27 5/3/99 Mon The Big Picture
F 5/10/99 Mon Final
 
Homework Assignments

#

Due

 Assignment Solution
1 1/28/99 Profit and Loss Curve Matlab
2 2/4/99 Bayes Rule, Data Snooping
3 2/11/99 TBA
4 2/18/99 TBA
5 2/25/99 TBA
6 3/4/99 TBA
7 ... ...
 
 

Comments
Please send all comments to aweigend@stern.nyu.edu.