Rikard König
email: rikard.konig@hb.se

Ulf Johansson
email: ulf.johansson@hb.se

Maria Riveiro
email: maria.riveiro@his.se

Peter Brattberg
email: peter.brattberg@hb.se

Modeling Golf Player Skill Using Machine Learning

Keywords: Classification, Decision Trees, Machine Learning, Golf, Swing analysis

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Golf is a major sport that is today played by over 60 million people all over the [START_REF] Wheeler | A Global Perspective on the Environmental Impact of Golf[END_REF].

The golf swing of a pro may look simple, but it is a combination of several complex biomechanical motions that need to be performed at high speeds with high accuracy. Hence, both pros and amateurs spend considerable time on perfecting their swings. However, due to the very complex chain of movements, the help of a teaching professional is most often needed to identify problems in a swing. Golf instructors require a deep theoretical knowledge about golf swings and a lot of first hand teaching experience to be able to teach at a high level. Naturally, finding the problem with a swing is a crucial skill that must be mastered. Since the golf swing is such a complex movement and since the club head moves at great speeds, golf swing analysis has long been an art requiring very sharp eyes. When determining the effectiveness of a swing, many teaching professionals often start by observing the ball flight and thereafter the club and body motion [START_REF] Smith | Professional golf coaches' perceptions of the key technical parameters in the golf swing[END_REF].

Lately, new technology, such as the TrackMan Launch Monitor Radar (TM) [START_REF]TrackMan A/S[END_REF], has made it possible to measure numerous characteristics of the golf swing quantitatively. TM units use a Doppler-radar to register information about the club head at the point of impact with the ball and the trajectory of the ball. In total, TM delivers 28 metrics, where eight are related to the club head and twenty are related to the ball flight.

Of course, technology like TM is a great tool for teaching professionals when analyzing swings. However, in practice, due to the complexity of the swing and the many metrics, teachers often focus on only a few parameters they consider the most important ones. In essence, the TM solves the problem of characterizing a swing quantitatively, but does not help to identify good and bad aspects of a particular swing.

Naturally, many previous studies have focused on analyzing the swing quantitatively using high speed video, e.g. see [START_REF] Fradkin | How well does club head speed correlate with golf handicaps?[END_REF] and [START_REF] Sweeney | The influence of club-head kinematics on early ball flight characteristics in the golf drive[END_REF]. However, due to the tedious manual labor related to video analysis, these and similar studies have only used a small number of players (20-45). A small number of example swings in combination with several metrics make an analysis of the interaction between different swing variables difficult. Hence, these and similar studies have been restricted to an analysis of single variables using statistical techniques.

By using a TM-unit, it is, however, feasible to collect quantitative data from a larger number of golfers, since all metrics are calculated automatically. More data enables the use of more powerful techniques, like machine learning, for modeling golf player skill. In this study, we evaluate whether it is possible to discern a good swing from a bad one by only using data from the club head at impact and from ball flight.

Background

Golf is a game in which the player aims to hit the ball from the tee to a hole in as few strokes as possible. A golf course normally consists of 18 holes, where each hole is designed to be played with a certain number of strokes. The number of intended strokes is called the par of the hole and ranges from 3-5 strokes. A golf course also has a par, calculated as the sum of the par of each hole, which is normally 72 strokes. Golf has a handicap system which is designed to let golfers with different levels of skill compete against each other. There are several different handicap systems of which the EGA [START_REF] Ega | EGA Handicap System[END_REF] and USGA are dominant. The EGA system is predominant in Europe and the USGA in the USA. In essence, a handicap system lets players deduct strokes according to their handicap (hcp). A hcp lies in the range of 36 to -4 (called +4). One simple way of calculating the final score using a hcp is to subtract the player hcp from the total number of strokes. When golfers play better than their hcp, e.g., the score minus the hcp is lower than the course par, their hcp is reduced a fraction and if they play worse it is increased. Hence, a player's hcp is supposed to be an estimation of that golfer's current skill level, where a better player has a lower hcp. Broadie in [START_REF] Broadie | Assessing Golfer Performance Using Golfmetrics[END_REF], sorted golf shots on a course into four different categories:

 Long game -shots longer than 100 yards.  Short game -shots shorter than 100 yards, not including sand shots.  Sand game -shots from bunker no longer than 50 yards.  Putting -shots on the green. To become a skilled player, all of these shots must be mastered, but according to Brodie, the long game is the most important one for amateur players' hcp.

The long game consists of shots from the tee and longer shots on the course. Tee shots can be hit with the driver, which is designed to hit the longest shots, or iron clubs, which are designed for different specific distances. Shots from the fairway are most often done using iron clubs.

Swings can be described in numerous ways using different terms; however, in this study, the terminology of the TM software is used consistently for simplicity. In total, Trackman delivers eight metrics related to the club head (CLUB):

 ClubSpeed -Speed of the club head at the instant prior to impact. Traditionally, the club path was believed to determine the starting direction and the face angle (opening or closing the club head) to be responsible for the curvature of the shot. Modern golf theory, however, has established that it is in fact mainly the face angle that determines the starting direction [START_REF] Tuxe N | The Secret of the Straight Shot II[END_REF]. Using Trackman terminology, the CLUB parameters, FaceAngle and ClubPath are the most important factors responsible for the starting direction of the ball, where FaceAngle is credited for 85% of the direction [START_REF] Tuxe N | The Secret of the Straight Shot II[END_REF]. Similarly, the curvature of the shot is a direct function of the difference between the ClubPath and the FaceAngle (FaceToPath), where a negative value represents a right-to-left movement, i.e., a draw for a right-handed golfer. Naturally, the aim in most cases, is to produce a fairly straight shot down the target line, which is achieved with a ClubPath and FaceAngle of approximately zero. The LaunchAngle of the ball is dependent on the DynamicLoft which, in turn, is related to the AttackAngle; a higher DynamicLoft will produce higher shots.

Golf theory maintains that the swings for iron clubs and those for the driver differ slightly; first, the driver is designed for longer shots and is hence longer than iron clubs, which results in higher ClubSpeeds. Secondly, when hitting with the driver, the ball is placed towards the front facing leg, instead of in the middle, as done with iron clubs. This is done to hit the ball on the way up of the swing, i.e., with a positive At-tackAngle, thus decreasing the DynLoft and thereby producing a longer shot due to a lower SpinRate. Shots with iron clubs are instead hit on the way down in the swing with a negative AttackAngle, in order to produce spin and thereby better control the length and direction of the shot.

Related work

Golf is a sport that many people are passionate about and, hence, much research regarding all aspects of the game has been carried out. However, not as much work based on quantitative data has been carried out, since the necessary technology has only become available in recent years. The following section presents a selection of some recent relevant studies where quantitative data was analyzed and related to players' skill.

In [START_REF] Fradkin | How well does club head speed correlate with golf handicaps?[END_REF] Fradkin Sherman and Finch performed a quantitative study of how the Club-Speed correlated with player hcp. Here, ClubSpeed speed was measured using high speed video and averaged over the strokes. Data was collected from forty-five male Australian golfers with hcps in the range of 2-27. The results show a very strong correlation (0.95) between hcp and club head speed.

Sweeny et al. performed another quantitative study in [START_REF] Sweeney | The influence of club-head kinematics on early ball flight characteristics in the golf drive[END_REF] and noted that even if many coaching and biomechanical texts describe how the kinematics of the club head at impact lead to distance and accuracy, there is limited quantitative evidence for these claims. Hence, an opto-reflective system was used to analyze the swings with the driver of 21 male golfers. Using the kinematics of the club at impact, i.e. Club-Speed, orientation path and centeredness, five kinematics of early ball flight, i.e., BallSpeed, LaunchAngle, LaunchDirection, SpinRate and SpinAxis, were modeled. Experiments show that these club kinematics could explain a significant part of the early ball flight, i.e., R 2 values between 0.71 -0.82 were achieved.

In [START_REF] Betzler | Variability in clubhead presentation characteristics and ball impact location for golfers' drives[END_REF] which is the only identified study that analyzed a larger group of players, 10 driver shots were recorded for 285 players. These shots were recorded mainly using five 1000Hz high speed cameras, but also the TM-launch monitor. The aim of this study was to evaluate the variability in club head presentation at impact and the resulting ball impact location on the club face, for a range of golfers with different hcps. The variability of ClubSpeed, SmachFactor, AttackAngle, ClubPath, FaceAngle and impact location was evaluated and compared between the different hcp groups. Statistical tests based on 10 shots from each player were used to show that overall players with lower hcps, i.e., players with hcp <=11.4, exhibited significantly less variation in all of the evaluated variables.

A rather different and interesting approach of evaluating golf player skill was taken in [START_REF] Broadie | Assessing Golfer Performance Using Golfmetrics[END_REF], where golf players registered real course shots in a database, using a computer. In total, 40.000 shots were registered from 130 different golfers. Each shot was then compared to a shot from a scratch player in the same situation. An interesting aspect of this study is that all parts of the games were analyzed, including the long game, short game, sand game and putting. The results show that for players with higher hcps, inconsistency was the main cause of a bad score, i.e. a few really poor shots often ruin the score. Another interesting result is that proficient players tend to be better at all parts of the game, but it is the long game, i.e. shots over 100 yards, that is the biggest influencing factor between low and high hcp players. What all of these studies have in common is that they evaluate each parameter separately using statistical methods. To summarize the results, it is good to have a high club head speed and low variability in the presentation of the club head at impact. It is, however, not clear how low variability in the club head can be achieved or how different club head parameters relate to each other. Hence, these results demonstrate little about the difference in the swings of players with low and high hcps.

Method

The aim of this paper is to explain what distinguishes really proficient players from amateurs and novices. An early design choice was to model player skill on the basis of the long game, which is supposed to be the most important hcp factor for less skilled players [START_REF] Broadie | Assessing Golfer Performance Using Golfmetrics[END_REF]. The long game consists of both driver and iron shots and, hence, it was decided to record shots from each player with both the driver and one iron club, i.e. the 7 iron (i7). The next crucial design choice was how to discern good players from those less skilled. Since the EGA hcp system is designed to account for the skill of a golfer, it was selected as the basis of skill estimation in this study. Figure 1 shows the handicap distribution of the 277 players participating in the study. The average hcp was 12.8; the majority of players (73%) were in the hcp range of 4-18, indicating a higher skill than normal, i.e. the mean male hcp in Sweden is 21.5 and 48% of players fall into the 4-18 range [START_REF] Sga | Swedish Golf Association[END_REF]. The EGA hcp system divides players into five different categories based on their hcp. The best category has hcp numbers equal to or lower than 4.5. Hence, this was first chosen as a group of proficient players. However, to make the dataset a little less unbalanced, this group was expanded to include golfers with hcps <=5. For simplicity, players with hcps <=5 are henceforth called Pros and players with higher hcps are called Hackers. In practice, this meant that 58 players, with hcps of 5 or lower, were classified as Pros and the rest (219 players) as Hackers.

Fig. 1. Handicap distribution among players

Data Collection

Most of the data was collected during 18 days in March, April and May. Due to a rather skewed sample with only a few Pros, another collection phase with only Pros was conducted, resulting in 15 additional players. All data was collected at the same training facility [START_REF]World of Golf[END_REF], where a special section was reserved for each day of data collection. Since the aim of this study was to find general patterns among larger groups of players, only male golfers hitting right-handed were targeted. Female and left to right stroking players only represented a small fraction of the players at the selected training facility and could hence not be recorded in sufficient numbers.

To record a player, the radar was positioned three meters behind and slightly to the right of the hitting mat. Next, the radar was aimed (using the Trackman Performance Studio software) at a flag approximately 250 meters straight in front of the hitting mat. Before a player was recorded, he was first allowed to warm up, in order to reach a comfortable stroking state. Thereafter, five consecutive strokes were recorded using the players own seven iron followed by five strokes with the driver.

Naturally, an identical setting with regard to temperature and wind would have been preferable. However, this was impossible, since an indoor facility was not available. Instead, TrackMan's built-in normalization functionality was used. When normalizing ball data, TrackMan utilizes information from the club head at impact to correct deviation caused by wind, temperature, altitude and ball type. Hence, the players were told to hit the balls in the direction of the flag using a normal full stroke, while disregarding any wind if present.

Preprocessing

Trackman Performance Studio output a total of 28 metrics, of which eight were related to the club head and twenty to the ball flight. Most metrics are measured directly, but some are calculated on the basis of other metrics. If the radar cannot measure some aspects accurately, which rarely happens but frequently enough to be an issue, it does not output any values. This typically occurs for club head related metrics, i.e. ClubSpeed, AttackAngle, ClubPath, SwingPlane and SwingDirection, at lower swing speeds. Ball related metrics are most often recorded properly, with the exception of SpinRate which is sometimes estimated instead or not reported at all. Of the 2780 shots that were recorded, 187 had at least one missing value. One approach of obtaining representative measurements, for a single player and club, could be to use the average value of all five recorded shots. If, however, one of the shots were to be a really poor one, the average values could be quite misleading. Instead, the more robust approach, argued by Broadie in [START_REF] Tuxe N | The Secret of the Straight Shot II[END_REF], of using the median value was chosen. The median value disregards both the worst and the best shots and should hence be a better estimate of normal standards. The question of how to define the median shot of a group of shots still remains. In this study, we chose the median shot based on LengthC of the ball, since the length of a shot is one of the most important aspects of a good shot. The distance from the target line, i.e. SideC, could, of course, be another alternative, but since a straight shot is not effective if it is not long enough, we settled for length in this study. In eleven cases, the median shot, i.e. the third longest shot, had some missing value. In these cases, the second best shot was used instead. If the second best longest also happened to have a missing value, the fourth best shot was used instead. For one single player all shots with the driver had at least one missing value and, hence, this player was removed from that dataset.

Since previous research has shown strong correlations between skill and consistency, the standard deviations of each of the 28 metrics were also calculated for each club using all five shots. Shots with missing values were not excluded, except in the calculation of the metric with the missing value. The average number of shots with no missing values was 4.7 for a specific club.

Another issue was how to best represent each metric for a predictive modeling technique. Most metrics, like LengthC, have a straightforward representation, but metrics related to angles need some extra consideration. FaceAngle is one example where the representation plays an important role, since the angle can be both positive and negative, i.e. representing an angle to the right or left of the target line. If no transformation is done, a big negative angle would be considered as smaller than a small positive angle. However, in relation to the target line, which is more relevant for the quality of a swing, the opposite is true. Hence, each metric related to the target line was replaced with two new variables, where the first one was the absolute value and the second was a binary variable with the same name but preceded with a P-, to represent whether the original angle was positive or not. Metrics related to vertical angles, i.e. AttackAngle, LaunchAngle, were not modified.

Experiments

The aim of this paper was mainly to investigate how well golf skill can be modeled using quantitative data from a TM-Radar and to understand which factors are the most important ones for a good golf swing. Naturally, this requires that the predictive models are transparent and comprehensible and, hence, WEKAs J48, which is an implementation of the famous decision tree algorithm C4.5 [START_REF] Quinlan | C4. 5: programs for machine learning[END_REF], was used in the main experiments. However, to evaluate how much predictive power the data actually contained, a Random Forest [START_REF] Breiman | Random forests[END_REF] was first applied as an upper benchmark.

The separation of the players into Pros and Hackers was rather unbalanced, i.e. 58 Pros vs 219 Hackers. Unbalanced data sets often result in models favoring the majority class at the cost of very few predictions for the minority class. Initial experiments confirmed that this was also the case for both the 7i-and the driver datasets in this study, resulting in true positive rates of around 35%. Hence, since the main goal was to model the proficient players, a cost of miss-classification equal to the unbalance rate was associated with the minority class. This was done for both RF and J48 using WEKA's meta cost sensitive classifier by assigning a cost of 219/58=3.78 for missclassifications of Pros. To keep the total weight of the instance to the original 277, WEKA, in practice, assigned a weight of 0.632 to Hackers and 2.389 to Pros.

When modeling using J48, the main goal is to describe good general swings in a comprehensible way. Hence, it was decided that at least 10% of the Pros, i.e. six players, should be present in any pure Pro leaf node. However, due to WEKAs reweighting of the instances, a minimum instance per leaf of 6 would only require three Pros. To avoid this and to ensure that in practice there were always at least six instances in a pure Pro leaf, the minimum number was instead set to 6 *2.389=14.

Another setting motivated by the class imbalance was to use Laplace estimates in J48, since previous studies, e.g. [START_REF] Provost | Tree induction for probability-based ranking[END_REF], have shown that it most often improves the probability estimates and thereby the ranking ability of the produced model.

To facilitate a comparison of the predictive power of variables related to club delivery and the ball flight, the original datasets were used to create four new ones based on four subsets of the original variables. The attribute subsets used to create the new datasets were:

 CLUB Variables related to the club delivery.  C-STD Player standard deviations for CLUB variables.  BALL Variables describing ball Launch and flight, Carry flat and Est. Total.  B-STD Player standard deviations for BALL variables. Based on these subsets, five datasets were then created; only CLUB, CLUB and S-STD, only BALL, Ball and B-STD and one final dataset containing all variables.

Model interpretation

Since all decision trees techniques optimize some kind of information gain criteria, starting at the root, splits closer to the root are normally considered more important. Information gain is the difference in purity, e.g. gini diversity index (GDI) [START_REF] Breiman | Classification and regression trees[END_REF] or entropy [START_REF] Quinlan | Induction of decision trees[END_REF], between the original dataset and the resulting subsets. The equation below defines a general way to calculate the information gain given a specific purity measure, where 𝑃(𝐷 𝑖 ) is the proportion of the dataset D that is placed in the subset Di. The split resulting in the highest purity gain is selected and the procedure is then repeated recursively for each subset in this split.

𝑔𝑎𝑖𝑛(𝐷, 𝑆) = 𝑝𝑢𝑟𝑖𝑡𝑦(𝐷) -∑ 𝑃(𝐷 𝑖 ) * 𝑝𝑢𝑟𝑖𝑡𝑦(𝐷 𝑖 ) 𝑠 𝑖=1

Consequently, the importance of the final splits can be, in the same way, evaluated by considering the number of training instances affected by the split and the resulting purity of the subsets.

Results

The following sections present the results of the experiments. First, basic statistics are presented followed by the predictive performance of the models. Finally, a selection of models are presented and interpreted, to ensure that they are practical and based on rational relationships.

Basic statistics

Table 1 shows the basic statistics of the players labeled as Hackers and Pros. For each attribute and club, the mean value and the standard deviation are presented. Note that the standard deviations are within each group and not for each player which is another variable used in the experiments discussed above. The most obvious differences between Hackers and Pros are in the values for Ball-Flight, CarryFlat and LengthT variables, i.e. Pros hit the ball longer and straighter. More specifically, Pros on average hit the ball 29 meters longer with the 7i and 49 meters longer with the driver. However, they also hit the ball straighter than the Hackers, (4 meters smaller deviation from the target line for both 7i and the driver), in spite of the longer shots. Note that since the Pros hit the ball substantially further, the difference in accuracy is essentially much greater than the 4 meter deviation implies. Furthermore, the standard deviations are much smaller (for all variables) for Pros, showing that they are a more homogeneous group, in terms of how they hit the ball.

Regarding the variables related to club delivery, the arguably biggest differences (always in favor of the Pros) concern ClubSpeed, AttackAngle ClubPath and FaceTo-Path, i.e. the Pros hit harder, more down on the ball (i7) and deliver the club and club face better in relation to the target line, resulting in longer and straighter shots. One variable that appear to be rather similar for the Hackers and Pros is the SwingPlane, which only differs by 1.6 degrees for the 7i and 2.1 for the driver.

The longer shots of the Pros can be further explained by the Ball Launch variables. Pros have a higher BallSpeed which is a result from the higher ClubSpeed and a better SmashFactor, which means that more energy is transferred from the club to the ball. Another interesting variable is the SpinRate, where a lower spin rate promotes longer shots while higher rates give more control over the ball flight. As could be expected, Pros have a lower SpinRate for the driver, thus maximizing distance, but a higher SpinRate for the 7 iron, thus enabling more control compared to the Hackers. Another basic assumption that can be verified is that it is common among Hackers to hit the ball with a slice or fade. Slice or fade is signified by a positive SpinAxis, which 69% of the Hackers have. The distribution of positive and negative SpinAxis among the Pros is even and SpinAxis itself is much smaller, resulting in just a small fade or draw. Fig. 2 presents the players in a scatter plot where the y-axis is the carry distance and the xaxis is the deviation in meters from the target line. As Table 1 also shows, Pros, in general, hit longer and straighter than Hackers. However, there are many Hackers who hit as far as the Pros and the longest shot (207m) was actually made by a Hacker (who, it transpired, was competing in the longest drive competition). The point is that some Hackers can easily be discerned on the basis of distance and side deviation, but far from all. The same pattern is apparent in Fig. 4, i.e. the group of Hackers has a greater variation in their SwingPlane and FaceAngle, however, only some can be easily discerned from the Pros using only these variables.

Predictive Performance

Table 2 below shows the results for RF, using n-fold cross validation for the different datasets. The first columns show the club that was used and the following four define the data that was present in each dataset. The first observation that can be made is that the results for datasets based on i7 and driver data are rather similar overall, even if they differ slightly for a specific dataset. Furthermore, BALL and B-STD are, in general, slightly better than the corresponding combination of CLUB and C-STD.

Using all the data, RF achieves a rather high accuracy of 86.6% for both clubs, while the naive approach of only predicting Hacker would achieve an accuracy of 79%. Due to the class imbalance, it is arguably more interesting to look at the area under the ROC curve (AUC), since it is a measure of how well the model can rank the players. Again, the RF performs well and reaches a high AUC of 0.901 for both clubs. The true positive rate (TP) for the Pro class is 72.4%, which is a reasonable level with the class imbalance in mind.

As may be expected, an analysis of the results for the attribute subsets shows that including the standard deviations for either the club or the ball flight provides extra predictive power. The second best results are achieved using datasets with BALL and B-STD, closely followed by CLUB and C-STD.

The data sets without any standard deviations, i.e. BALL and CLUB for i7 and driver, all clearly result in models with lower predictive performance. Using just BALL data, compared to BALL and B-STD, results in a drop of 8.6% in TP for 7i and 12% for the driver. Similarly, using only CLUB data results in a loss in TP of 12% for i7 and 26% for the driver, compared to using CLUB and C-STD.

Next, the results of J48 are presented in Table 3. Compared to the results of RF in Table 2, it becomes apparent that RF is superior for all datasets and metrics, except for a few cases where J48 achieves a higher TP. The general results for J48 are also similar to those of RF, where the datasets containing the standard deviation of the ball (STD-B) again give the best results, regardless of the club that was used. This also correlates with previous research showing that consistency is a very important aspect of the swing. For both clubs, the best result is achieved using all variables, closely followed by only BALL and B-STD. The da-tasets containing only CLUB and C-STD variables were slightly worse, but still surprisingly accurate. An interesting detail is that adding STDs to the CLUB data only had a minimal effect for the 7i but a rather large one for the driver.

The most interesting result is, however, that the J48 achieves a rather high predictive performance with relative small and simple models. Hence, it can be argued that the trees capture a substantial part of what distinguishes good swings from bad ones while still being small and comprehensible. The predictive power is, of course, higher for RF, but these models are opaque and hence of limited value for human inspection and analysis. Next, we present a few sample J48 models to evaluate how well they fit with existing golf theory and how usable they would be in a teaching situation.

Interpretation of models

The decision trees presented below were created using data from all the players, but do otherwise use an identical setup to the main experiment. Hence, the overall accuracy of these models for new players should be judged on the results in Table 3.

The result for each leaf is, however, still presented, to show the accuracy of each rule for the existing golfers. The trees are presented in textual form where each leaf is given a number preceded with # to facilitate referencing. Leaves defining Pros and splits leading to Pro leaves are marked in green, while Hacker leaves and splits are marked in red. Splits that do not directly lead to a leaf are marked in black, since they do not define a property exclusively for Pros or Hackers. Each leaf also reports the number of golfers of the leaf class, the true positive rate (TP) of the leaf and the average hcp of all players classified by the leaf, i.e. (#players of leaf class|TP|hcp). Four trees, one based on all the data and one based on only CLUB data for each club, are discussed below. The reason for this choice is to present the most accurate trees (using all the data) and the more usable trees based on only club data.

The general result of the interpretation of the models below is that they also concur with general golf theory. Another general result is that Hackers can be identified with less splits than Pros. In practice, this means that you must master several skills to become a Pro, while just a few deficiencies in the swing are often enough to categorize a hacker. Another interesting observation is that all the trees tend to group players with similar hcps in a leaf, thus creating an implicit ranking of the players, in spite of the problem being setup as a classification task. This signifies that each split has predictive power in itself and thus represents a skill that needs to be mastered by a golfer. The contrary would apply if all the leaves relating to a particular class (Hacker or Pro) would have a similar average hcp. This would mean that all the splits leading to a rule would be needed to discern Hackers from Pros. Looking at this explains the higher AUC which exactly measures the ability to rank the players.

Model based on all data for i7

Next, Fig. 5 below, depicts a tree created from all the recorded data for the i7, which resulted in the most accurate model. Looking at the particular tree presented in Fig. 5, the first group of Hackers, i.e. rule #5, is signified by having a high STD in their launch direction. In practice, this means that they have problems controlling the starting direction of their shots. The average hcp for this large group of 115 Hackers is 17.

The next group of Hackers classified by rule #1 (62 Hackers with an average hcp of 14.7) is better at controlling the LaunchDirection but has a lower DistHeight than the better golfers in leaf #3-#4. A lower DistHeight means that they do not hit the ball far enough, i.e. the apex and the ball trajectory are not as far away from the player, which naturally results in a shorter shot. The first group of Pros, the largest group of 39 Pros with an average hcp of 4.2, is discerned by leaf #2. These Pros hit the ball further than the Hackers in group #1 and have a lower STD of their SmashFactor than players in groups #3 and #4. In essence, this means that they are more consistent in how well they hit the ball, which of course is important for controlling the length and accuracy of a shot.

The last two groups (#3 and #4) have a higher STD in their SmashFactor and a higher average hcp (7.4 for #3 and 11 for #4). Rule #3 defines the better golfers of these two groups which are characterized by their ability to better control the STD of their DynLoft. The DynLoft has a very strong correlation to the launch angle, which is an important factor for DistHeight and thereby the length of the shot. Hence, the difference between the Pros in rule #3 and the Hackers in rule #4 is that the Pros are better at controlling the length of their shots. The difference in hcp is, however, not large, which explains the low TP of .52 for rule #3.

To summarize, this tree tells us that the first skill a golfer must learn is to start the ball consistently on the target line and thereafter to hit long enough. Once mastered, the next skill to conquer is the ability to consistently hit the ball the same distance.

Models based on CLUB data for i7

Even if this tree, presented in Fig. 5, is one of the more accurate trees, it does not say much about how the swing itself should be performed, since this is mainly based on BALL-data. Hence, a model for 7i based only on CLUB-data.

The first split in this tree is based on the DynLoft with a split point of 29.6. This value is far from the Pros' mean value of 23.5 and the average Hacker value of 25.3 presented in Fig. 6. Hence, the 24 players that are all correctly classified as Hackers by leaf #7 have a way to high dynamic loft. The next group of players discerned by leaf #1 is again Hackers with a more reasonable DynLoft but a rather low ClubSpeed of 77.1, resulting in shorter shots. It is not a big difference from the average Club-Speed for Hackers, which is 80.2, but far from the Pros at 87.2. Leaf #5 shows that the next group of Hackers has high FaceAngle, resulting in shots in the wrong direction, if there is no corresponding curvature of the ball (hook or slice) to counteract the ini-tial direction. Nonetheless, swings with higher FaceAngles are harder to control and repeate consistently.

Players in group #5 are again classified as Hackers who are here discerned from the remaining golfers by a rather large ClubPath angle (4.8) in relation to the target line. A high ClubPath value will also make it harder to control the ball flight, for more or less the same reasons as the FaceAngle. It is, however, interesting to note that FaceAngle is used higher up in the tree, which signifies that it is a more important factor than the ClubPath. This concurs with modern golf theory which has only been applied a few years.

Leaf #4 is the first leaf to classify players as Pros. Players classified by this rule (including 37 Pros and 24 Hackers) have avoided the rules leading to the leaves #1,#7,#6,#5 and thus have a reasonable swing, but also a high ClubSpeed, i.e. higher than 86.2. Players who have avoided the leaves #1, #7,#6,#5, but have a lower Club-Speed than the Pros in #4, are classified as Pros or Hackers by leaves #2 and #3, depending on the SwingPlane. Here, a lower swing plane is favorable, which by itself is interesting, especially since Table 1 does not show any major difference in the swing plane between Hacker and Pros.

It should be noted that even if the TP rate for leaf #2 is only 0.41, it still classifies the players as Hackers due to the cost-sensitive learning applied. Nonetheless, the average hcp of this leaf is lower than all the Hacker leaves and 5 points lower than the neighboring Hacker leaf (#3). Hence, the split is still important for discerning Hackers from Pros.

Discussion and conclusions

From the results presented above, it is clear that it is possible to model golf player skill on the basis of quantitative data from the club head at impact or the ball flight.

Predictive models with high ranking ability, i.e. AUC, can be created using both RF and J48 with an advantage to RF. For the 7i datasets containing only CLUB data, there was no difference in the predictive performance of the models. Obviously, including STDs and ball data facilitates more complex relationships, which the random forest can model but not the decision tree. The models do, nonetheless, concur in the variables they consider important, which adds credibility to the found models.

It is also interesting to note that the ranking ability is high, in spite of a setup as a binary classification task. When interpreting the models, this can be seen in the fact that players in leaves longer from the root, in general, have a lower average hcp. Obviously, each split has predictive power and leaves further from the root require that more skills need to be fulfilled by a player.

Even if the random forests are more accurate, they do not give insights as actionable as the decision trees. Future work could however address this deficiency by applying a sensitivity analysis showing the relative importance of the variables. The relative importance would possibly give a teaching professional sufficient knowledge on the particular aspect of the swing that they should improve to lower a player's hcp. This is an important point, since teaching professionals are normally mainly concerned with improving a player's swing more ideal, which does not always result in a lower hcp, the goal of many Hackers.

All J48 models are, however, small, comprehensible and concur with modern swing theory. More specifically, consistency is more important than any specific value of the club head or ball flight. Looking at only the CLUB variables, it is clear that the ClubSpeed, FaceAngle, ClubPath and DynLoft are the most important variables for discerning good players from those less skilled. This also concurs with golf theory and is hence a strong result for the correctness of the models.

A more surprising and interesting result is, however, that the SwingPlane is also a strong indicator of skill, where proficient players tend to have flatter swings. This is surprising, since there is only a small deviation in the average SwingPlane of Hackers and Pros. The reason for this result is not obvious from modern golf theory and we can only speculate about the reasons; e.g., a flatter swing could possibly lead to higher ClubSpeed at the bottom of the swing, due to a more circular movement of the club. However, both decision tree models created from only CLUB data contradict this theory, since they split the players on the basis of ClubSpeed before SwingPlane. Hence, the SwingPlane is used to discern among players with rather similar Club-Speed. Naturally, the flatter swings found at better players in this study is an important finding that needs to be studied in more detail. Obviously, a flatter plane gives rise to a different chain of movements in the swing, thus affecting the release pattern and the impact position. Specifically, a flatter swing plane is associated with swinging from the inside and a release leading to a dynamic loft and higher smash factor, which was found to be favorable in the induced models.

Finally, the J48 models show a clear distinction in the swing of Hackers and Pros. When modeled using only CLUB data, the swings of golfers classified as Pros were characterized by reasonable values for FaceAngle, ClubPath and DynLoft while having a high ClubSpeed. The majority of Hackers could be identified by a large deviation (from the average Pro value), for one of these variables. Better Hackers were discerned by having either a slightly lower ClubSpeed or a steeper SwingPlane than the Pros.



  AttackAngle -Vertical movement of the club through impact.  ClubPath -Horizontal movement of the club through impact. (+) = inside-out, (-) = outside-in.  SwingPlane -Bottom half of the swing plane relative to the ground.  SwingDirection -Bottom half of the swing plane relative to the target line.  DynLoft -Orientation of the club face, relative to the plumb line, at point of impact (POI).  FaceAngle -Orientation of the club face, relative to the target line, at POI. (+) = open face, i.e., for a right-handed golfer to the right of the target line (-) = closed face.  FaceToPath -Orientation of the club face, relative to the club path, at POI. (+) = open path, i.e., for a right-handed golfer to the right of the club path (-) = closed path. In addition to these CLUB-related metrics, TM also registers twenty metrics related to the ball flight (BALL):  BallSpeed,BallSpeedC -Ball speed the instant after impact, speed at landing.  SmashFactor -Ball speed / club head speed at the instant after POI.  LaunchAngle -Launch angle, relative horizon, immediately after impact.  LaunchDirection -Starting direction, relative to the target line, of the ball immediately after impact. (+) = right, (-) = left.  SpinRate -Ball rotation per minute the instant after impact.  SpinAxis -Tilt of spin axis. (+) = fade / slice, (-) = draw / hook.  VertAngleC -Ball landing angle, relative to the ground at zero elevation.  Height, DistHeight, SideHeight -Maximum height of shot at apex, distance to apex, apex distance from the target line.  LengthC, LengthT -Length of shot, C = calculated carry at zero elevation, T = calculated total including bounce and roll at zero elevation.  SideC, SideT -Distance from the target line, C = at landing, T = calculated total including bounce and roll. (+) = right, (-) = left. Access to launch monitors producing quantitative data, and a consensus about impacting variables, have allowed a revision of swing theory. Most importantly, im-proved analysis has produced a fundamental change in the understanding of the ball flight.

Fig. 2 .

 2 Fig. 2. i7 -Carry Length vs. Carry side Next, Fig. 3 instead shows the standard deviation of the five recorded 7i-shots for each player. Obviously, the Pros are more consistent, i.e. deviate less, than the Hackers, in spite of hitting the ball further. Again there is no clear distinguishing boundary separating the two classes, except for the really substandard Hackers.

Fig. 3 .

 3 Fig. 3. i7 -STD of Carry Length vs. Carry side

Fig. 4 .

 4 Fig. 4. i7 -SwingPlane vs. Face Angle

Fig. 5 .

 5 Fig. 5. Tree based on i7: CLUB, STD-C, BALL and STD-B

Table 1 .

 1 Basic statistics for Hackers and Pros

			i7				Driver
		Mean	STD		Mean	STD
		Hacker	Pro	Hacker Pro Hacker	Pro	Hacker Pro
	Club Delivery
	ClubSpeed	80.2	87.2	8.6	5.9	94.5	104.7	10.7	7.7
	AttackAngle	-3.2	-4.8	2.4	2	-1.7	-1.1	3.5	2.9
	ClubPath	4	2.9	3	2.4	3.8	3.6	3.2	2.7
	P-ClubPath	0.45	0.6	0.5	0.49	0.49	0.69	0.5	0.46
	SwingPlane	61.9	59.3	4.4	3.6	49.9	47.8	4.5	3.4
	SwingDir.	4.8	3.5	3.4	2.6	5.6	4.8	4.1	3.6
	P-SwingDir.	0.35	0.26	0.48	0.44	0.42	0.53	0.49	0.5
	DynLoft	25.3	23.3	4.8	2.9	14.3	12.6	4.9	2.7
	FaceAngle	3.3	2.1	2.6	1.6	4.2	2.6	3.1	1.8
	P-FaceAngle	0.38	0.52	0.49	0.5	0.33	0.45	0.47	0.5
	FaceToPath	4	2.5	3.4	1.7	4.6	3.4	3.7	2.4
	P-FaceToPath	0.42	0.38	0.49	0.49	0.33	0.24	0.47	0.43
	Ball Launch
	BallSpeed	104	116.1	11	6.7	133.1	151.8	16.5	10.4
	SmashFactor	1.3	1.33	0.06	0.04	1.41	1.45	0.07	0.04
	LaunchAngle	20	18.4	3.8	2.8	12.3	11.2	4.2	2.7
	LaunchDir.	3	2	2.3	1.7	3.8	2.6	2.8	1.7
	P-LaunchDir.	0.4	0.55	0.49	0.5	0.34	0.47	0.48	0.5
	SpinRate	5470	5686	1532	847	2837	2207	1318	613
	SpinAxis	6.1	3.3	5	2.3	10.7	8.2	7.8	5.4
	P-SpinAxis	0.69	0.5	0.46	0.5	0.6	0.47	0.49	0.5
	Ball Flight
	DistHeight	86.8	102.7	14.8	8.5	118	148.4	26.2	18.5
	Height	24.7	30.1	7.2	5.7	19.6	22.8	9	6.6
	SideHeight	4.8	3.2	4.1	2.9	8.1	6.4	6.3	5.1
	P-SideHeight	0.45	0.53	0.5	0.5	0.39	0.47	0.49	0.5
	Carry Flat
	LengthC	142.1	163.3	21.1	11.4	192.2	238.1	36.8	25.8
	SideC	9	5.7	7.9	5.1	15.6	12	12	10.1
	P-SideC	0.46	0.5	0.5	0.5	0.44	0.47	0.5	0.5
	VertAngleC	42.3	45.9	6.7	4.6	29	29.2	9.9	6.5
	BallSpeedC	54.7	56.6	3.1	2.4	65.6	68.6	7.7	4.6
	Est. Total
	LengthT	153.2	172.2	21.2	11.3	223.3	271.8	36.4	24.8
	SideT	9.1	5.5	8.1	4.9	17.5	13.2	13.3	11.1
	P-SideT	0.46	0.5	0.5	0.5	0.46	0.5	0.5	0.5

Table 2 .

 2 Random forest n-fold results

	RF	CLUB	C-STD	BALL	B-STD	ACC	AUC	TP
	i7	X				0.773	0.792	0.534
	i7	X	X			0.848	0.886	0.655
	i7			X		0.798	0.833	0.69
	i7			X	X	0.874	0.901	0.776
	i7	X	X	X	X	0.866	0.907	0.724
	Driver	X				0.801	0.824	0.569
	Driver	X	X			0.797	0.872	0.638
	Driver			X		0.826	0.839	0.655
	Driver			X	X	0.855	0.902	0.776
	Driver	X	X	X	X	0.866	0.908	0.724

Table 3 .

 3 J48

	n-fold results

  To summarize, there are two groups of Pros who all have reasonable DynLoft and FaceAngle. The best groups of Pros are, in addition, characterized by a high Club-Speed. The other group of Pros has a lower ClubSpeed but they distinguish themselves from the Hackers by a flatter SwingPlane. FaceAngle > 7.26 #5: Hacker (29|1.0|19.6) DynLoft > 18.26 #6: Hacker (41|1.0|19.2)

	Fig. 8. Tree based on Driver: CLUB

Models based on all data from the driver

In the following section, models based on data from shots with the driver are presented. Fig. 7 depicts a decision tree created using all the available data. The first group of Hackers, i.e. leaf #6 in Fig. 7, is selected on the basis of the STD of the LaunchAngle (2.89), which lies close to the Pros value of (2.7). Consequently, a large number of golfers, (93) are selected in this leaf but, surprisingly, all are correctly classified as Hackers. Next, leaf #5 selects 27 Hackers on the basis of a relatively large STD of the FaceAngle. More Hackers are classified in leaf #1 on the basis of a BallSpeed less than 128.5, which is rather low compared to the average BallSpeed of Pros at 151.8. Obviously, these players cannot hit the ball far enough.

Leaf #4 is characterized by a higher STD of the SmashFactor, which relates to how consistently the player hits the ball. The split point of 0.04 is identical to the Pros mean value and, consequently, a few Pros (3) are miss-classified as Hackers in this leaf. The final split divides Pros from Hackers on the basis of their SwingPlane, where it is again favorable to have a flatter plane. The chosen split point of 51.89 is higher than the average driver SwingPlane of both Pros and Hackers, thus discerning rather good Hackers with atypical swings, i.e. the average hcp of rule #3 is 9.8.

S-LaunchAngle

Models based on CLUB data from driver

The final tree presented in Fig. 8 is based on only CLUB variables for the driver. This tree selects exactly the same subset of variables as the i7 tree presented in , but in a slightly different order. The split point values are, however, quite different; which is natural because the driver is a longer club and designed to launch the ball at a lower angle.

The first group of Hackers in leaf #6 has a DynLoft at more than 18.26 degrees, which is much too high, compared to the mean values of the Pros which are at 12.6 degrees. A high DynamicLoft results in a higher LaunchAngle and thereby a higher and shorter shot. Next, leaf #5 discerns players with a high FaceAngle, i.e. 5 degrees higher than that of the Pros, which results in shots in the wrong direction or curved shots that are harder to control.

The third group, defined by leaf #4, is the first to classify golfers as Pros on the basis of a ClubSpeed of 104.14, which is very close to the average value for Pros. Since the leaves #5 and #6 are based on rather extreme values, the main factor discerning these really proficient players, with a mean hcp at 5.5, from the rest of the golfers is high swing speed and reasonable values for FaceAngle and DynLoft. Of the remaining golfers, 44 players are classified as Hackers in leaf #3, on the basis of a rather steep swing plane. Finally, the remaining players are classified by leaves #1 and #2, again on the basis of their ClubSpeed. Players with club speeds lower than 89.54, which is very low compared to Pros at 104.7, are classified as Hackers and the rest as Pros. The TP rate for this last set of Pros is low (0.38), but due to the cost-sensitive training the leaf still classifies golfers as Pros. It could, however, be argued that the predictive power of this leaf is higher than indicated by this value, since the average hcp of this rule is 4.5 points lower than any leaf classifying Hackers. Obviously, players misclassified by this rule are still rather good players.


Saurabh Nagrecha
email: snagrech@nd.edu

Pamela Bilo Thomas
email: pthomas4@nd.edu

Keith Feldman
email: kfeldman@nd.edu

Nitesh V Chawla
email: nchawla@nd.edu

Predicting Chronic Heart Failure Using Diagnoses Graphs

Keywords: heart failure, cardiovascular disease, directed acyclic graph, Medicare, EMR, health care

Predicting the onset of heart disease is of obvious importance as doctors try to improve the general health of their patients. If it were possible to identify high-risk patients before their heart failure diagnosis, doctors could use that information to implement preventative measures to keep a heart failure diagnosis from becoming a reality. Integration of Electronic Medical Records (EMRs) into clinical practice has enabled the use of computational techniques for personalized healthcare at scale. The larger goal of such modeling is to pivot from reactive medicine to preventative care and early detection of adverse conditions. In this paper, we present a trajectory-based disease progression model to detect chronic heart failure. We validate our work on a database of Medicare records of 1.1 million elderly US patients. Our supervised approach allows us to assign likelihood of chronic heart failure for an unseen patient's disease history and identify key disease progression trajectories that intensify or diminish said likelihood. This information will be a tremendous help as patients and doctors try to understand what are the most dangerous diagnoses for those who are susceptible to heart failure. Using our model, we demonstrate some of the most common disease trajectories that eventually result in the development of heart failure.

Introduction

Today the healthcare industry finds itself at the precipice of a significant change, as the past decade has seen the adaption and integration of electronic medical records (EMR) into clinical practice. Beyond the logistical benefits of maintaining and organizing patients' medical data, clinicians and researchers can perform novel research using these secondary data sources [START_REF] Prokosch | Perspectives for medical informatics[END_REF][START_REF] Peter B Jensen | Mining electronic health records: towards better research applications and clinical care[END_REF][START_REF] Belle | Big data analytics in healthcare[END_REF]. In fact EMRs ability to provide a computationally accessible set of structured data representing the expansive healthcare feature space has fueled the emergence of a sundry of informatics tools ranging from early clinical decision support systems, to the statistical analysis of, to predictive analytics aimed at identifying patients at risk for readmission [START_REF] Kawamoto | Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success[END_REF][START_REF] Oanh Kieu Nguyen | Predicting all-cause readmissions using electronic health record data from the entire hospitalization: Model development and comparison[END_REF].

Building on the success of these tools, many researchers have seen healthcare informatics as the junction between another line of parallel clinical research, the shift from reactive to preventative medicine. Medical research is itself an evolving field, and has advanced in parallel with the emergence of EMR. Clinicians have put forth a strong effort in advancing the care paradigm from reactive medicine, where clinicians treat the conditions currently afflicting a patient, to preventative care where clinicians undertake courses of action "for the purpose of preventing disease or detecting it in an asymptomatic stage" [START_REF] Stanislav | Health behavior, illness behavior and sick role behavior: I. health and illness behavior[END_REF]. As such, the early detection and treatment of adverse health conditions represents an exciting opportunity for the informatics community. Others have found that a combination of research areas, including, but not limited to, graph-based data mining, entropy-based data mining, and topological-based data mining, work best for knowledge discovery and towards an end goal of supplementing human learning with machine learning [START_REF] Holzinger | Trends in interactive knowledge discovery for personalized medicine: Cognitive science meets machine learning[END_REF]. Eventually the goal is to have P4-medicine (predictive, preventative, participatory, personalized) available for all patients by using big data and the combined human computer interaction and knowledge discovery/data mining approach [START_REF] Holzinger | Trends in interactive knowledge discovery for personalized medicine: Cognitive science meets machine learning[END_REF].

A number of works have built on this foundation, focusing predictive tasks from disease prediction, to the prediction of breast cancer survivability [START_REF] Delen | Predicting breast cancer survivability: a comparison of three data mining methods[END_REF][START_REF] Darcy A Davis | Time to care: a collaborative engine for practical disease prediction[END_REF]. However, these tools suffer from a fundamental flaw, they identify patients' health conditions as isolated events, i.e. a disease will occur in a patient's future medical chart, or a patient will recover from early stage breast cancer. One must remember that an individuals' health condition does not only consist of when doctors measure them in a clinical environment. Although the rate of onset may vary, the progression of disease represents a highly fluid state. As such, it may be more valuable to view these patients' conditions as trajectories, rather than binary events.

While this seems like a significant shift in thinking, medical subfields have already established the concept of a disease trajectory, sometimes denoted as disease 'progression'. In particular, research in relation to neurodegenerative disorders such as Parkinson's and Alzheimer's have quite well established this concept [START_REF] Marek | The parkinson progression marker initiative (ppmi)[END_REF][START_REF] Wilkosz | Trajectories of cognitive decline in alzheimer's disease[END_REF]. More recently, the trajectory concept has begun expanding into the general healthcare population. Many clinicians have long postulated that an underlying progression of related diagnoses may relate to diagnoses for which we do not explicitly relate a temporal aspect. Today, the data collected through the expanding EMR now allows for researchers to examine such hypotheses in detail. Perhaps Jensen et. al, have provided one of the best examples to date, where through their work they successfully extracted diagnosis trajectories by analyzing millions of longitudinal patient records and utilized a novel way of describing biological disease progression [START_REF] Boeck Jensen | Temporal disease trajectories condensed from population-wide registry data covering 6.2 million patients[END_REF].

In this work, we build on this concept and present a novel graph-based diagnosis trajectory model. While recent advances have taken what effectively represent an "unsupervised" approach to trajectory discovery, we aim to provide a target based "supervised" methodology. We will begin with a discussion of the underlying methodology used in constructing the underlying diagnosis graph. From here, we will discuss utilizing the temporal relations extracted from the graph, showing that we can identify paths that significant differentiate the occurrence of the target diagnosis. Finally, we will provide a case study of the methodology in relation to patients with congestive heart failure.

Data Description

Electronic Medical Records (EMRs) log information on patients in the form of diagnosis codes for each of their visits. This log effectively narrates a patient's medical history as identified by medical practitioners and can predict their future health outcomes. Here we describe our data source, the data structure, how chronic heart failure appears these diagnoses logs and how prevalent it is within our patients.

Provenance Our data comes from the Medicare records of 1,145,541 elderly patients in the United States. The accuracy and completeness of these records makes them invaluable to demographic and epidemiological research [START_REF] Lauderdale | Epidemiologic uses of medicare data[END_REF][START_REF] Mitchell | Using medicare claims for outcomes research[END_REF][START_REF] Darcy A Davis | Time to care: a collaborative engine for practical disease prediction[END_REF]. The data is completely anonymized -both in terms of the patients and the healthcare providers. For a given patient, we applied a threshold of a maximum of 5 in-patient visit, and each visit corresponds to a maximum of 10 diagnosis codes from the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). These ICD-9-CM codes are designed to convey an intrinsic hierarchy of diagnosis detail-the full 5-digit code represents the specific condition, location and/or severity, and its leading 3 digits represent the medical diagnosis family. This "code collapse" [START_REF] Darcy A Davis | Time to care: a collaborative engine for practical disease prediction[END_REF] helps us identify the family of patients who develop heart failure in our data.

Identifying heart failure. We observe ground truth evidence of presence/absence of heart failure with ICD-9-CM diagnoses for individual patients. Diagnoses represented by the family of '428.xy' codes cover all diagnoses for heart failure. Specific examples of the 428 diagnosis family include Systolic heart failure (428.2); which breaks down into Systolic heart failure, unspecified (428.20), Acute systolic heart failure (428.21), Chronic systolic heart failure (428.22) and Acute on chronic systolic heart failure (428.23). We labeled as 'HF' all patients for whom we observed the '428' diagnosis family, and labeled the rest as 'NHF' for heart failure and non-heart failure respectively.

Table 1 shows a sample patient's medical history. Here we see the chronological history of the patient through each successive visit expressed in terms of full ICD-9-CM codes. For each visit, the first code is the principal diagnosis, followed by any secondary diagnoses made during that visit. The data presents these diagnoses in their full ICD-9-CM form where 733.00 represents Osteoporosis, unspecified. Some diagnoses, such as Pathologic fracture (733.1) and Nutritional marasmus (261), use fewer than the maximum 5 digits in ICD-9-CM. patients and predict whether they will develop heart failure and validate this against observed ground truth data about these test patients.

Building a Representational Predictive Model

Researchers have built contemporary disease progression models using patient data with already known target outcomes. [START_REF] Marek | The parkinson progression marker initiative (ppmi)[END_REF][START_REF] Wilkosz | Trajectories of cognitive decline in alzheimer's disease[END_REF]. In contrast, our supervised approach helps contextualize disease progression trajectories against an in-situ control set of patients, i.e. our data contains trajectories followed by patients who eventually were diagnosed with Heart Failure and those who were not. This approach highlights the diagnosis trajectories that intensify or diminish likelihood of heart failure in patients. The identification of such divergence in diagnoses helps pinpoint signals for heart failure from overall population trends. In this section, we describe how we restructure Medicare EMR data to obtain supervised disease progression trajectories. We then merge individual trajectories in the form of a compact Directed Acyclic Graph to model class-aware patient population-wide trends in diagnoses. Using this model, we identify key differentiating diagnoses and trajectories that help separate patients who are likely to develop heart failure from those who do not.

Preprocessing. We transform the data from raw medical histories shown in Table 1 to extract class-aware trajectories for patients using the following steps. We first collapse the diagnosis codes to their 3 digit counterparts, then eliminate duplicate families of diagnoses and then decouple the diagnosis history used for prediction from the observed outcome. Table 2 shows the result of applying this preprocessing to the example history from Table 1.

Removing patients who receive a heart disease diagnosis on their first visit

Out of the 46.6% of the patients in our dataset who develop heart failure, 18.0% receive a heart failure diagnosis in their very first visit. Since this study revolves around the concept of diagnoses leading up to heart failure, we consider these patients out of scope for our training and testing data. This removal of heart failure cases reduces the rate of observed heart failure in the rest of the patients down to 34.8% from the original 46.6%. 2. Decoupling input data and target labels-In patients with heart failure, we right-censor the diagnosis data when the first '428' code appears. This ensures that there is no "data leakage", i.e. we do not predict heart failure based on an observed diagnosis of heart failure since it is a chronic condition. 3. Pre-pruning diagnoses and pathways-To mitigate the impact of spurious/noisy disease trajectories in our analysis, we set a minimum support threshold of 100 for the nodes and edges in our graph. By imposing this threshold, we ensure that none of the diagnoses or the pathways between them draws conclusions from a set of fewer than 100 patients out of a total sample size of 1.1M patients. 4. Code Collapse-The original data contains 12,396 "Minor Category" diagnosis codes, whereas our analysis targets the "Major Category" outcome (heart failure). Collapsing the 5-digit diagnoses codes down to their respective 3-digit major categories helps reduce the complexity of the problem and matches the granularity of the observed outcome. As a result, we now use 1,064 diagnosis families to chart patient trajectories, which is 8.6% of the original complexity. 5. Removing duplicate diagnoses-We only consider new and previously unobserved diagnoses in our analyses. In Table 1, this means that we consider diagnoses for Other disorders of bone and cartilage (733) only for their first V (Supplementary classification of factors influencing health status and contact with health services) and E (External causes of injury) reveal little about the progression of disease and were taken out of the graph.

Disease Progression for Individual Patients. For the example patient in Table 2, we can now create a disease progression history based on their diagnoses Figure 1. Each node represents a diagnosis and each edge (e(i, j)) represents a potential transition from diagnosis i to j across successive patient visits. Each of these edges is strictly directed from a diagnosis in visit (t -1) to a diagnosis in visit t and nodes in the same visit do not have edges between them. This makes the graph a Directed Acyclic Graph (DAG). Each patient in our training data has an outcome label associated with them, which we use as a label for each of these patient-centric DAGs.

A Class-Aware Heart Failure Model. While each patient's disease progression DAG contains signals for their eventual outcome, but a domain expert aided causal analysis for each patient would not scale to the 1.1M patients in our dataset. Instead, we aggregate these histories into a unified model across all patients to get a consensus on which diagnoses and pathways signal which outcome for an unseen patient. Within our patient population, we perform a 5-fold cross-validation, training on 80% of the patients and testing the model on the other 20% over 5 folds of the data. As shown in Figure 2, we simply combine observed nodes and edges across all training set patients and create a composite DAG. The nodes at each level represent the superset of possible diagnoses at that visit and edges between each level represent the observed transitions between diagnoses. The weight of each edge e(i, j) corresponds to the observed confidence of heart failure among patients who were diagnosed with i and then j. Similarly, we assign a weight to each node n representing the observed confidence of heart failure for that node. A patient can have multiple diagnoses within the same visit, each of which adds to the support of the corresponding nodes and edges. This does not guarantee the total incoming support into a node being equal to the total outgoing support. Another salient feature of this model is that it cane distinguish the same diagnosis code between visits. For example, if one observes code '261' in visit #1 and #2 for different patients, we create nodes labeled '261 1' and '261 2' to preserve their unique trajectory histories. By this definition, it is possible for '261 1' and '261 2' to have completely different weights. In our model, we label each node's diagnosis code according to the visit it was observed in.

Model Inference

The overall trained model is an interconnected representation that contains 1,974 nodes and 26,229 edges. with an average in-degree and outdegree of 13.29. A relatively few nodes and edges contain a high likelihood of heart failure as seen in Figure 4. These high-confidence nodes and edges indicate underlying diagnoses and trajectories that lead to high rates of heart failure. We identify nodes and edges with a high propensity for heart failure in Table 3 and Table 4 respectively. These nodes and edges describe diagnostic pathways that indicate heart failure. In addition to extremely high likelihood of heart failure, we also identify diagnoses that effectively discern heart failure. For this, we use information gain (or InfoGain, for short) for successive diagnoses in patients. A higher information gain indicates a higher class polarization between heart failure and non-heart-failure. Information Gain (IG) is the reduction in entropy for a given edge in our model. Specifically, an edge e(i, j) with high information gain indicates that the diagnosis of j after i leads to a higher confidence of arriving at either class. We compute each node's entropy from its class distribution using

H(i) = k -p k (i)log(p k (i)) H(j) = k -p k (j)log(p k (j)) IG(i, j) = H(i) -H(j)
where i and j are the respective source and destination diagnoses, k ∈ {HF, N HF }, and p k (.) represents the probability of observing class k in a given node. Information Gain for e(i, j) is simply IG(i, j) = H(i) -H(j), where higher values of IG are more helpful in our search for heart failure propensity intensifying markers.

The sheer abundance of pathways towards non-heart-failure outcomes eclipses the relatively low InfoGain of individual edges which intensify heart failure. These are the majority of the edges which form the positive side of Figure 5a. However, we are primarily interested in edges which intensify likelihood of heart failure. To achieve this, we artificially penalize InfoGain in edges which have a higher likelihood of non heart failure by simply making them negative. This isolates and highlights heart-failure intensifying pathways in the network. The above steps outline how we process raw patient records into a supervised representational model. This graphical model not only amalgamates patient disease trajectories, but it also highlights key pathways leading to heart failure.

Predicting heart failure for an Unseen Patient

Now that we have a representational and interpretable model to predict heart failure, we use it to predict outcomes for our held-out test dataset. We describe how we convert a new patient's diagnosis history into predicted probabilities and how we evaluate these predicted outcomes.

How to Predict Given a test patient's diagnosis history, we replicate the steps in the training section to arrive at a graph similar to Figure 1. Here, we make an important assumption about the nature of our model-we assume that the probabilities at each stage obey a Bayesian model. Using the probabilities from our trained model, we can predict relative odds of heart failure and non-heart failure by simply multiplying the class-wise probabilities for each edge and normalizing them. Given a test patient with a disease progression graph G test , the unnormalized value of P(Y = HF ) * = e p(e(i, j)), ∀e(i, j) ∈ G test and p(e(i, j)) ∈ G trained . We then similarly compute the unnormalized value of P(Y = N HF ) * and finally output the normalized value of P(Y = HF ).

In this work, we assume a Markovian model when using the Bayesian network structure to model disease progression. This means that dependencies and graph attributes (for instance, support and confidence) do not extend beyond immediate descendants directly in our model, i.e. A|B and B|C can model disease progression, but not A|B, C directly. In future work, the model can extend to include higher-order dependencies [START_REF] Austin R Benson | Higher-order organization of complex networks[END_REF]. This would enable us to model dependencies of the form A|B, C and beyond.

Evaluation & Baselines We compute the above probabilities for all patients in the test set and evaluated against their true observed outcomes. We then compute the Receiver Operating Characteristics in terms of False Positive Rate and True Positive Rate for these predictions. Our key prediction metric is the area under the ROC curve (AUROC), a higher AUROC indicating superior predictive performance.

How soon can we predict Heart Failure? Each visit in our trained model is represented by a layer of nodes in the DAG. A prediction made using the first i layers of the DAG corresponds to a prediction made on i visits of an unseen test patient. Deliberately pruning the number of layers in the trained DAG is equivalent to reducing the complexity of our trained model and being able to predict our target outcome earlier. In order to see if this trade-off negatively influences predictive performance, we test the unseen patient histories on DAGs pruned to predict heart failure from 1, 2, 3 and 4 (maximum number of test visits in our data) visits and evaluate their area under the ROC curve.

Results

The techniques described above cover three key aspects of our research. First, we train a class-aware model of heart failure from patient history data. Second, we interpret the model to identify key diagnoses and disease progression pathways which intensify or mitigate the chances of a given patient developing heart failure. Third, we show how this model performs when predicting heart failure outcomes for a completely unseen set of patients.

Model Inference Looking at the highest confidence nodes for heart failure in Table 3, several common themes appear in diagnoses that tend to proceed heart failure -namely, rheumatic fever, pulmonary congestion, cardiomyopathy, blood poisoning, kidney disease, hypertension, and aortic and mitral valve disease. For these diagnoses, it does not appear to matter much if one diagnoses a patient on 1 or 2 -the progression to heart disease seems to occur at about the same confidence levels. By absolute numbers in this data set, the diagnoses that lead to heart failure the most are cardiomyopathy and aortic mitral valve diseases.

Rheumatic heart diseases and pulmonary congestion patients appear less than the former three in the data set, but have a higher probability of leading to heart failure.

Referring to the highest confidence edges for heart failure given in Table 4, we can see that may of the same destination diagnoses match the diagnoses given in the high confidence nodes in Table 3. The edges give us some idea about the diagnoses that come first that may lead to heart disease given another diagnosis. For instance, the high confidence nodes in Table 3 told us that diagnoses such as rheumatic fever, pulmonary congestion, and cardiomyopathy lead to heart failure. 14 out of the 20 top confidence edges involve cardiomyopathy, which indicates that cardiomyopathy is a strong component in leading to heart failure. Cardiac dysrhythmia is a diagnosis that is particularly deadly when combined with further diagnoses.

The findings of this graph seem to confirm the results of other studies. Others have identified cardiomyopathy and valve dysfunction as precursors for heart disease [START_REF] John | Epidemiology, aetiology, and prognosis of heart failure[END_REF], [START_REF] Mark | Kidney disease as a risk factor for development of cardiovascular disease[END_REF], [START_REF] Lloyd-Jones | Heart disease and stroke statistics-2010 update[END_REF]. The American Heart Association has recommended that patients who have chronic kidney disease are in the highest risk group for development of cardiovascular disease [START_REF] Mark | Kidney disease as a risk factor for development of cardiovascular disease[END_REF]. Researchers have associated nephrotic syndrome with cardiovascular disease [START_REF] Mark | Kidney disease as a risk factor for development of cardiovascular disease[END_REF]. Pulmonary congestion is very common in patients with heart failure due to its relation to high pressure in the left ventricle of the heart. Many patients who have heart failure are also found to have fluid overload which is a common result of pulmonary congestion. Detection and treatment of pulmonary congestion can help prevent progression of heart failure [START_REF] Picano | Why, when, and how to assess pulmonary congestion in heart failure: pathophysiological, clinical, and methodological implications[END_REF]. The result of blood poisoning and sepsis is often multiple organ failure, including septic cardiomyopathy, which can lead to heart failure [START_REF] Laszlo | Complementrelated molecular events in sepsis leading to heart failure[END_REF]. Other studies have found that chronic pulmonary heart disease is a predictor of chronic heart failure in China [START_REF] Ym Cao | A pilot survey of the main causes of chronic heart failure in patients treated in primary hospitals in china[END_REF]. For many years, doctors have known that rheumatic fever can contribute to heart failure occurring later in life [START_REF] Edward | Rheumatic fever and rheumatic heart disease[END_REF]. Hypertension is also a major contributing factor in congestive heart failure [START_REF] Levy | The progression from hypertension to congestive heart failure[END_REF]. Almost all the top confidence edges involved rheumatic heart disease, pulmonary congestion and hypostasis, mitral and aortic valve disease, or cardiomyopathy, which therefore accentuates the importance of those diseases in the diagnosis of heart failure. The high confidence edges given in Table 4 let us know that diseases such as cardiac dysrhythmia, diabetes, ischemic heart disease, and lung diseases, diagnosed beforehand, can ultimately result in heart failure.

Table 5 shows us the highest information gain nodes tend to come from source diagnoses that are mental or noncardiac in nature (Affective psychoses, cerebral degeneration, malignant neoplasm of bladder, Parkinson's disease, etc.) followed by an acute myocardial infarction, endocardium diseases, or cardiomyopathy. This seems to suggest that these diagnoses are the first cardiac problems that might occur in patients with other mental or noncardiac issues. This model of information gain suggests that screening for those three diseases, since they appear as some of the first cardiac diagnosis on a trajectory that leads to heart failure. Heart Failure Prediction Our model can predict heart failure in patients from diagnoses from their second visit (i.e.: their first disease progression) as seen in Figure 6. Adding diagnoses from subsequent visits makes the predictive performance plateau in comparison to the second visit. As discussed in our Data Preprocessing stage in Section 3, we only have a maximum of 4 patient visits for our prediction task.

Discussion

Which disease progression trajectories lead to heart failure? A diagnosis of cardiomyopathy is a very common theme that appears in many high confidence edges and high information gains that lead to heart failure. Cardiomyopathy appears in 14 out of the 20 high confidence edges and 5 of the top 20 high information gains. We can therefore conclude that cardiomyopathy is an important factor in the progression of a heart failure trajectory. Monitoring patients for cardiomyopathy and intervening early is therefore important in limiting heart disease.

Besides cardiomyopathy, most of the other high information gain edges had a destination edge of acute myocardial infarction or endocardium diseases. These three appear as "gateway" diagnoses that eventually results in heart failure later in their medical record for patients who do not currently have a diagnosis of heart disease.

While cardiomyopathy is very common in the high confidence edge nodes, it does not occupy the top four high confidence edges. Cardiac dysrhythmia appears as a source diagnosis in the two top confidence edges, indicating that those with cardiac dysrhythmia should watch out for rheumatic heart disease or pulmonary congestion. Additionally, pulmonary congestion appears as a destination node for three out of the top four confidence edges, indicating that pulmonary congestion is a complication that, given other diagnoses such as cardiac dysrhythmia, diabetes, or chronic ischemic heart disease, could eventually result in heart failure.

Can we predict heart failure? Using this model, we observe we can predict heart failure using the conclusions found from this data. The ROC curve given in Figure 6 indicates that diagnoses given in the first visit contains most of the information that leads to heart failure. We receive diminishing returns from subsequent diagnoses after that first visit. One reason could be that most of the diseases that eventually result in heart failure have already appeared by their first visit to a doctor, and rarely do patients not have diseases that are indicative of heart failure at their first visit, and then they go on to get heart failure later. Table 5 gives some examples of patients that have the highest jump in the probability of developing heart failure after their first visit. Certain cardiac events put those who were originally being treated for mental diagnoses in particular (affective psychoses, cerebral degenerations, Parkinson's disease, neurotic disorder) on a path to heart failure beginning in Visit 2. In general, the data tells us that the disease progression from Visit 1 to Visit 2 gives the most indication that a patient will eventually become a heart failure patient.

Conclusion

By constructing a DAG of Medicare patients and their visits, we found trends in diseases that result in an ultimate diagnosis of heart failure. We conclude that cardiomyopathy is a condition that is commonly associated with heart failure such that screening for cardiomyopathy should be a common part of preventative treatment. Additionally, we know that many patients' first diagnoses on a heart failure path are acute myocardial infarctions, endocardium diseases, and cardiomyopathy. Doctors who see patients for other medical issues, especially mental issues as observed, should know of these complications since they are often the first that show up in diagnoses that do not otherwise lead to heart failure. We also found that rheumatic heart disease, pulmonary congestion and hypostasis, cardiomyopathy, blood poisoning, and valve and aortic diseases are common comorbidities that occur before doctors diagnose patients with heart failure. Because the highest information gains in our DAG are on paths that concern mental disorders such as psychosis, cerebral degeneration, and Parkinson's, the conclusion can be made that patients being seen for these disorders should also be monitored for heart disease.

The ultimate goal of such a system is to be able to effectively predict likelihood of heart failure, which we demonstrate using our trained DAG. We show that the most indicative diagnoses belong to the first disease progression in terms of their information gain and area under the ROC curve. This underscores the usefulness of our model in extracting signals which can be used for early detection of heart failure.

Fig. 1 :

 1 Fig. 1: Example Patient Disease Progression A directed acyclic graph can model a single patient's disease progression as shown above. Each layer represents a distinct visit, and each child node belongs to the next visit. In each visit layer, we identify several unique diagnosis codes. Each individual node in the graph represents these diagnosis codes.

Fig. 2 :

 2 Fig. 2: Combining individual Patient Histories

Fig. 3 :

 3 Fig. 3: Degree Distribution of Trained Diagnosis Graph

Fig. 4 :Fig. 5 :

 45 Fig. 4: Heart failure trajectories are highly imbalanced: We observe that the progression of heart failure follows a minor set of nodes and edges in the learned model.

  Fig-ure 5b shows how this transformation affects the edge attributes and isolates the relatively few edges which exhibit a high InfoGain favoring heart failure.

Fig. 6 :

 6 Fig.6: Predictive performance for various number of visits. Area under the ROC curve plateaus with an increased amount of visits.

Table 1 :

 1 Example Patient History: Each row represents a distinct visit in chronological order. In each visit, the data shows multiple ICD-9-CM code diagnoses for our example patient. Note that the code for heart failure (428.0) appears in the fifth visit.

	Visit	Vector of ICD-9-CM Disease Codes
	1 7331 (Pathologic fracture, unspecified site), 73300 (Osteoporosis, unspec-
	ified), 2761 (Hyposmolality and/or hyponatremia), 4928 (Other emphy-
	sema), 73743 (Scoliosis associated with other conditions)
	2 7331 (Pathologic fracture, unspecified site), 73300 (Osteoporosis, unspec-
	ified), 73741 (Kyphosis associated with other conditions), 73743 (Scol-
	iosis associated with other conditions), 261 (Nutritional marasmus)
	3 7331 (Pathologic fracture, unspecified site), 73300 (Osteoporosis, unspec-
	ified), 73741 (Kyphosis associated with other conditions), 73743 (Scol-
	iosis associated with other conditions), 261 (Nutritional marasmus)
	4 485 (Bronchopneumonia, organism unspecified), 2765 (Volume depletion
	disorder), 2769 (Electrolyte and fluid disorders not elsewhere classified),
	496 (Chronic airway obstruction, not elsewhere classified),73300 (Osteo-
	porosis, unspecified)
	5 48230 (Pneumonia due to Streptococcus, unspecified) 4280 (Heart fail-
	ure), 5119 (Unspecified pleural effusion), 2761 (Hyposmolality and/or
	hyponatremia), 2768 (Hypopotassemia), 73300 (Osteoporosis, unspeci-
	fie), 73741 (Kyphosis associated with other conditions), 7331 (Pathologic
	fracture, unspecified site)
	Using Table 1 as an example, we see that in visit #5, the patient was diagnosed
	with Congestive heart failure, unspecified (428.0) and therefore belongs to the
	class 'HF'.
	Summary Statistics. The EMR data used in this study covers 1,145,541 el-
	derly Medicare patients over the course of 5,727,705 total visits. Over the course
	of these visits, the patients registered a total of 12,396 unique ICD-9-CM di-
	agnoses codes, which represent 1,064 families of 3 digit collapsed codes. This
	set of patients exhibits a heart failure rate of 46.6%, which is extremely high
	compared to the United States, about 5.7 million (2.2%) adults have heart fail-
	ure [15]. However, some have observed the overall prevalence of heart failure
	in elderly patients in the United States as high as 10.6 to 13.5% (Chart 20-
	2 [15]). Since our study focuses on patients on Medicare, this number is further
	amplified.
	Experiments. Based on this EMR data, we group our analysis into two distinct
	phases-1) building a representational model for heart failure and 2) predicting
	heart failure outcomes for unseen patients. First, we infer the nature of disease
	progression for patients with and without observed heart failure. Based on the
	learned model, we identify trajectories, individual diagnoses, and edges that give
	the best indication of heart failure. We then use this model on previously unseen

Table 2 :

 2 Preprocessed Example Patient History. Using the same example data as Table1, we derive a compact history of the patient. As a result of this preprocessing, we arrive at a set of input diagnoses (visits #1 to 4) to create a trajectory and decouple it from the labeled outcome (visit #5). Note that the diagnosis has been collapsed to its family

	Visit #	ICD-9-CM codes
	1	733 (Other disorders of bone and cartilage), 276 (Disorders of fluid elec-
		trolyte and acid-base balance), 492 (Emphysema), 737 (Curvature of
		spine)
	2	261 (Nutritional marasmus)
	3	NA
	4	485 (Bronchopneumonia, organism unspecified), 496 (Chronic airway ob-
		struction, not elsewhere classified)
	5	428 heart failure

Table 3 :

 3 Top 20 confidence nodes for heart failure. These nodes represent the diagnoses with the highest likelihood of heart failure in our model.

	Visit # Diagnosis	Confidence Support
	2	Rheumatic fever with heart involvement	0.5947	882
	1	Rheumatic fever with heart involvement	0.5685	690
	2	Pulmonary congestion and hypostasis	0.5117	2088
	1	Cardiomyopathy	0.4923	7338
	1	Diseases of mitral valve	0.4746	1872
	2	Cardiomyopathy	0.466	11864
	1	Pulmonary congestion and hypostasis	0.4552	1046
	1	Poisoning by agents primarily affecting the	0.4448	679
		cardiovascular system
	1	Hypertensive heart and renal disease	0.4383	3784
	2	Hypertensive heart and renal disease	0.4366	7624
	1	Diseases of mitral and aortic valves	0.4312	8724
	1	Diseases of aortic valve	0.4232	199
	2	Diseases of mitral valve	0.4188	2764
	2	Diseases of aortic valve	0.4097	487
	1	Diseases of other endocardial structures	0.4032	3524
	1	Chronic pulmonary heart disease	0.4016	4924
	1	Nephrotic syndrome	0.3963	1676
	2	Diseases of mitral and aortic valves	0.3949	15168
	2	Chronic pulmonary heart disease	0.3918	9477
	1	Hypertensive heart disease	0.375	41999

Table 4 :

 4 Highest Confidence disease progression edges in trained DAG. Top 20 diagnostic edges with extremely high confidence of heart failure. These edges represent those at the extreme right of the distribution in Figure 4b. Visit number corresponds to source diagnosis.

	Visit #	Source Diagnosis	Visit #	Destination Diagnosis		Conf Supp
	1	Cardiac dysrhythmias	2	Rheumatic	fever	with	0.6995 183
				heart involvement
	1	Cardiac dysrhythmias	2	Pulmonary congestion and	0.6870 131
				hypostasis
	1	Diabetes mellitus	2	Pulmonary congestion and	0.6798 178
				hypostasis
	1	Other forms of chronic is-	2	Pulmonary congestion and	0.6486 148
		chemic heart disease		hypostasis
	1	Other diseases of lung	2	Cardiomyopathy		0.6400 150
	1	Cardiomyopathy	2	Pneumonia, organism un-	0.6209 153
				specified
	1	Cardiomyopathy	2	Acute myocardial infarc-	0.6198 121
				tion
	1	Conduction disorders	2	Cardiomyopathy		0.6123 325
	1	Acute myocardial infarc-	2	Cardiomyopathy		0.6000 180
		tion
	1	Old myocardial infarction	2	Cardiomyopathy		0.5986 147
	1	Cholelithiasis	2	Cardiomyopathy		0.5943 106
	1	Cardiomyopathy	2	Other diseases of lung		0.5882 170
	1	Diverticula of intestine	2	Cardiomyopathy		0.5847 118
	1	Cardiomyopathy	2	Conduction disorders		0.5738 237
	1	Cardiac dysrhythmias	2	Cardiomyopathy		0.5724 1277
	1	Cardiomyopathy	2	Transcient cerebral is-	0.5714 105
				chemia
	1	Ill-defined descriptions and	2	Cardiomyopathy		0.5708 212
		complications of heart dis-
		ease
	1	Diabetes mellitus	2	Chronic pulmonary heart	0.5706 340
				disease
	1	Essential hypertension	2	Pulmonary congestion and	0.5610 164
				hypostasis
	1	Iron deficiency anemias	2	Cardiomyopathy		0.5577 104

Table 5 :

 5 Top 20 Information Gain edges in trained DAG. These edges represent diagnoses which go from seemingly benign to high likelihood of heart failure.

	Visit #	Source Diagnosis	Conf. Visit #	Destination Diagnosis	Conf.	InfoGain Supp
	1	Affective psychoses 0.1541	2	Acute myocardial in-	0.3462 0.1597 153
						farction
	1	Affective psychoses 0.1541	2	Other diseases of en-	0.3355 0.1544 225
						docardium
	1	Affective psychoses 0.1541	2	Hypertensive heart	0.3279 0.1533 210
						disease
	1	Malignant neoplasm	0.1769	2	Acute myocardial in-	0.3462 0.1489 106
		of bladder				farction
	1	Other cerebral degen-	0.1717	2	Acute myocardial in-	0.3462 0.1324 105
		erations				farction
	1	Other cerebral degen-	0.1717	2	Other diseases of en-	0.3355 0.1272 194
		erations				docardium
	1	Other cerebral degen-	0.1717	2	Hypertensive heart	0.3279 0.1261 141
		erations				disease
	1	Parkinson's disease 0.1795	2	Acute myocardial in-	0.3462 0.1252 151
						farction
	1	Parkinson's disease 0.1795	2	Other diseases of en-	0.3355 0.1200 167
						docardium
	1	Parkinson's disease 0.1795	2	Hypertensive heart	0.3279 0.1189 161
						disease
	1	Neurotic disorder	0.1874	2	Acute myocardial in-	0.3462 0.1068 249
						farction
	1	Neurotic disorder	0.1874	2	Other diseases of en-	0.3355 0.1016 286
						docardium
	1	Neurotic disorder	0.1874	2	Hypertensive heart	0.3279 0.1005 247
						disease
	2	General symptoms	0.2176	3	Cardiomyopathy	0.2980 0.1004 152
	2	Other disorders of	0.2208	3	Hypertensive heart	0.3109 0.1004 146
		urethra and urinary			and renal disease
		tract
	2	Other disorders of	0.2208	3	Cardiomyopathy	0.2980 0.0980 199
		urethra and urinary
		tract
	1	Senile and presenile	0.1898	2	Acute myocardial in-	0.3462 0.0979 179
		organic	psychotic			farction
		conditions
	1	Senile and presenile	0.1898	2	Other diseases of en-	0.3355 0.0926 221
		organic	psychotic			docardium
		conditions
	2	Other and unspeci-	0.2282	3	Hypertensive heart	0.3109 0.0917 207
		fied anemias			and renal disease
	1	Senile and presenile	0.1898	2	Hypertensive heart	0.3279 0.0915 225
		organic	psychotic			disease
		conditions