510 likes | 903 Views
SEKALA LIKERT DALAM ………. METODE SURVEI KAJIAN LINGKUNGAN. FOTO: smno.kampus.ub.febr2012. Likert Scale . Dr. Rensis Likert. A Likert scale ( /ˈ lɪkərt / ) is a psychometric scale commonly involved in research that employs questionnaires.
E N D
SEKALA LIKERT DALAM ………. METODE SURVEI KAJIAN LINGKUNGAN FOTO: smno.kampus.ub.febr2012
Likert Scale Dr. RensisLikert A Likert scale ( /ˈlɪkərt/) is a psychometric scale commonly involved in research that employs questionnaires. It is the most widely used approach to scaling responses in survey research, such that the term is often used interchangeably with rating scale, or more accurately the Likert-type scale, even though the two are not synonymous. The scale is named after its inventor, psychologist RensisLikert. Likert distinguished between a scale proper, which emerges from collective responses to a set of items (usually eight or more), and the format in which responses are scored along a range. Technically speaking, a Likert scale refers only to the former. The difference between these two concepts has to do with the distinction Likert made between the underlying phenomenon being investigated and the means of capturing variation that points to the underlying phenomenon. When responding to a Likert questionnaire item, respondents specify their level of agreement or disagreement on a symmetric agree-disagree scale for a series of statements. Thus, the range captures the intensity of their feelings for a given item, while the results of analysis of multiple items (if the items are developed appropriately) reveals a pattern that has scaled properties of the kind Likert identified. Diunduhdari: http://en.wikipedia.org/wiki/Likert_scale
What is Likert scale? • It is a psychometric scale commonly involved in research that employs questionnaires. • It is the most widely used approach to scaling responses in survey research. • Likert scales are a non-comparative scaling technique and are one-dimensional in nature. • When responding to a Likert questionnaire item respondents specify their level of agreement or disagreement on a symmetric agree-disagree scale for a series of statements. • Thus, the range captures the intensity of their feelings for a given item, while the results of analysis of multiple items reveals a pattern that has scaled properties of the kind Likert identities SKALA LIKERT Paling banyakdigunakanuntukpengukuranperilaku Skala yang terdiridaripernyataandandisertaijawabansetuju-tidaksetuju, sering-tidakpernah, cepat-lambat, baik-burukdsb. (tergantungdaritujuanpengukuran). C. Bird menyebutnya Method of Sumated Ratings SKALA LIKERT DIGUNAKAN PADA SAAT: Menggambarkanposisi RELATIF individudalamkelompoknya Membandingkanskorsubyekdengankelompoknormatifnya Diunduhdari: http://www.google.co.id/search?q=varimax+rotation&num=10&hl=id&source=lnms&sa=X&ei=_pk1UPCBEMvLrQf3y4GYBA&sqi=2&ved=0CAUQ_AUoAA&biw=1272&bih=506#hl=id&sclient=psy-ab&q=skala+likeret+ordinal&oq=skala+likeret+ordinal&gs_l=serp.3..0i13j0i13i30l3.39393.46279.0.47075.24.24.0.0.0.15.1278.9326.0j2j7j8j2j1j2j1.23.0...0.0...1c.9-oocxdIH4k&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&fp=3322fb4802ff5da6&biw=1272&bih=531………….. 24/8/2012
What is Likert scale? SKALA LIKERT DIGUNAKAN PADA SAAT: Menggambarkanposisi RELATIF individudalamkelompoknya Membandingkanskorsubyekdengankelompoknormatifnya Menyusunskalapengukuran yang sederhanadanmudahdibuat. SkalaLikert Skalalikertadalahskala yang mengukursikapdenganmenyatakansetujuatauketidaksetujuanterhadapsubyek, obyekataukejadiantertentu. Urutanuntukskalainiumumnyamenggunakan lima angkapenilaianyaitu (1). Sangattidaksetuju (2) setuju (3) Netral (tidakpasti) (4) Tidaksetuju (5) SangatTidakSetuju. Urutanitubisadibalik. Alternatifangkabisabervariasidari 3 sampaidengan 9 Diunduhdari: http://www.google.co.id/search?q=varimax+rotation&num=10&hl=id&source=lnms&sa=X&ei=_pk1UPCBEMvLrQf3y4GYBA&sqi=2&ved=0CAUQ_AUoAA&biw=1272&bih=506#hl=id&sclient=psy-ab&q=skala+likeret+ordinal&oq=skala+likeret+ordinal&gs_l=serp.3..0i13j0i13i30l3.39393.46279.0.47075.24.24.0.0.0.15.1278.9326.0j2j7j8j2j1j2j1.23.0...0.0...1c.9-oocxdIH4k&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&fp=3322fb4802ff5da6&biw=1272&bih=531………….. 24/8/2012
Five – point Likert item Likert Scale Difference Likert item The Likert scale is a summative scaling technique developed by RensisLikert in the 1930s. Likert scales are typically 5-point rating scales ranging from "Strongly Agree" through "Neither Agree nor Disagree" to "Strongly Disagree." An extensive list of possible statements regarding attitudes to particular research question is generated by researchers and the respondent indicates the extent to which he/she agrees with the statement. Likert scale Disadvantages : Likert scaling is quite tricky to get right. The researcher must be able to prove that each item of the questionnaire has a similar psychological 'weight' in the respondent's mind, and that each item is making a statement about the same construct. psychometric validation. If this is not accomplished the results of the scale will be unreliable. Likert scales tend to produce "ceiling effects", where a group may rate items close to the upper limit of the scale Diunduhdari: ………….. 23/8/2012
LIKERT ITEM Likert item is considered symmetric or balanced because there are equal amounts of positive and negative positions. Often five ordered response levels are used, although many psychometricians advocate using seven or nine level, a recent empirical study found that a 5 or 7 point scale. Stems and Scales A familiar method for assessing attitudes is the Likert item. A Likert item consists of two parts: a stem, which is simply a statement of an attitude, and a scale on which people express their agreement with that statement. For example: Stem: I believe that capital punishment is cruel. Scale: Disagree strongly Disagree somewhat Can't say Agree somewhat Agree strongly Diunduhdari: http://www.actualanalysis.com/likert.htm ………….. 24/8/2012
LIKERT ITEM A five-point scale of agreement, like the one above, is probably most common, but longer or shorter scales can be used. Shorter scales are more difficult to get useful information from, though. Likert items are easy to use and they can give you useful information. A single item, however, will rarely give you any useful information, so you are best to use sets of them. Ideally you would use 40 items or more, but useful information can be got with fewer. A single item rarely provides useful information simply because responses to it are affected by many factors in addition to the one you're interested in. When several items are used, the consistency of responding produced by an attitude can be detected. Diunduhdari: http://www.actualanalysis.com/likert.htm ………….. 24/8/2012
The format of a typical five-level Likert item Sample question presented using a five-point Likert item An important distinction must be made between a Likert scale and a Likert item. The Likert scale is the sum of responses on several Likert items. Because Likert items are often accompanied by a visual analog scale (e.g., a horizontal line, on which a subject indicates his or her response by circling or checking tick-marks), the items are sometimes called scales themselves. This is the source of much confusion; it is better, therefore, to reserve the term Likert scale to apply to the summed scale, and Likert item to refer to an individual item. A Likert item is simply a statement which the respondent is asked to evaluate according to any kind of subjective or objective criteria; generally the level of agreement or disagreement is measured. It is considered symmetric or "balanced" because there are equal amounts of positive and negative positions. Often five ordered response levels are used, although many psychometricians advocate using seven or nine levels; a recent empirical study found that a 5- or 7- point scale may produce slightly higher mean scores relative to the highest possible attainable score, compared to those produced from a 10-point scale, and this difference was statistically significant. In terms of the other data characteristics, there was very little difference among the scale formats in terms of variation about the mean, skewness or kurtosis. Diunduhdari: http://en.wikipedia.org/wiki/Likert_scale
The format of a typical five-level Likert item 1. Strongly disagree2. Disagree3. Neither agree nor disagree4. Agree5. Strongly agree The format of a typical five-level Likert item, for example, could be: Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree
CONTOH: Q.18.Please measure the following affirmative perceptions about your library OPAC and Web OPAC use .
Q.19. Please mark your appreciation towards the use of your library OPAC/Web OPAC. Diunduhdari: ………….. 23/8/2012
The format of a typical Seven-level Likert item INTERPRETASI SKOR SKALA LIKERT Tidakdapatdilakukansecaralangsung Harusdibandingkandenganskorkelompoknormatifnya Diunduhdari: ………….. 23/8/2012
TAHAPAN PENYUSUNAN SEKALA LIKERT TAHAPAN PENYUSUNAN (1) Menentukandanmemahamidenganbaikapa yang akandiukur Menyusun Blue Print untukmemandupenyusunanalatukur Indikator yang secarateoritis-logismemberikontribusi yang lebihbesarharusdiberikanpernyataan yang lebihbanyak Pernyataandibuat Favorable dan Unfavorable. TAHAPAN PENYUSUNAN (2) Membuat Item sesuaidengankaidah Ujicoba item Memilih item yang baik Menyusun item terpilihmenjadisatu set alatukur Menginterpretasikanhasilpengukuran. Diunduhdari: ………….. 23/8/2012
TAHAPAN PENYUSUNAN SEKALA LIKERT MEMILIH PERNYATAAN (1) Memilihdengannilai t, denganlangkah: Menghitungdanmenjumlahkanskortiapsubyek Mengelompokkansubyekmenjadidua. Menggunakan mean atau median jikasubyeksedikit, danmenggunakanpercentil 25 - 75 atau 30 - 70 apabilasubyekbanyak MEMILIH PERNYATAAN (2) Menghitungnilai t denganrumus: Diunduhdari: ………….. 23/8/2012
TAHAPAN PENYUSUNAN SEKALA LIKERT MEMILIH PERNYATAAN (3) Pilihlah 20 – 25 item dengannilai t yang tinggidansemuaindikatorharusterwakilioleh item Favorable danUnfovorable Nilai minimal t yang baikadalah 1,75. MEMILIH PERNYATAAN (4) Memilihdengannilai r (korelasi), denganlangkah: Menghitungdanmenjumlahkanskortiapsubyek Mengkorelasikanskortiap-tiap item denganskor total yang diperolehsetiapsubyek. Diunduhdari: ………….. 23/8/2012
TAHAPAN PENYUSUNAN SEKALA LIKERT MEMILIH PERNYATAAN (5) Nilai r hitungdibandingkandengan r tabel. Pilihlah item yang r hitungnyapositifdanlebihbesardari r tabel Biasanyadapatjugamenggunakanpatokan r minimal 0,3 Buang item yang r hitungnyakurangdari r tabelataukurangdari 0,3 danhitungkembalikorelasinyahingga r hitungsemua item lebihdari r tabelataulebihdari 0,3 Pilihlah 20 – 25 item dengannilai r yang tinggidansemuaindikatorharusterwakilioleh item Favorable danUnfovorable. MENYUSUN PERNYATAAN MENJADI SATU SET SKALA Penyusunan item terpilihdalamsatu set skalaharusacakberdasarkanindikatormaupun item Favorable dan Unfavorable. Diunduhdari: ………….. 23/8/2012
METODE ANALISIS • Depending on how the Likert scale questions are treated a number of different analysis methods can be applied • Analysis methods used for individual questions (ordinal data) • Bar charts and dot plots • Not histograms (data is not continuous) • Central tendency summarised by median and mode • Not mean • Variability summarised by range and interquartile range • Not standard deviation • Analysed using non-parametric tests • (difference between the medians of comparable groups) • Mann- whitney U test • Wilcoxon signed –rank test • Kruskal – wallis test Diunduhdari: ………….. 23/8/2012
METODE ANALISIS • 2. When multiple Likert question responses are summed together (interval data) • All questions must use the same Likert scale • Must be a defendable approximation to an interval scale (i.e. coding indicates magnitude of difference between items but there is no absolute zero point) • All items measure are single latent variable (i.e. a variable that is not directly observed, but rather inferred from other variables that are observed and directly measured) • Analyzed using parametric tests • Analysis of variance (ANOVA) KegunaandanAsumsi One Way ANOVA digunakanuntukmengujiperbedaan rata-rata lebihdariduasampel. Asumsi-asumsi One Way ANOVA: • Populasi yang akandiujiberdistribusi normal. • Variansdaripopulasi-populasitersebutadalahsama. • Sampeltidakberhubungansatudengan yang lain. Diunduhdari: openstorage.gunadarma.ac.id/handouts/S1.../SLIDE-PE-1.ppt ………….. 24/8/2012
METODE ANALISIS Analysis of Variance (ANOVA) ANOVA 1 Arah Desain 2 Faktor Dgn. Replikasi Desain Blok Lengkap Acak Uji-F Uji-F Uji Tukey- Kramer Uji Perbedaan Signifikan Fischer Terkecil Diunduhdari: openstorage.gunadarma.ac.id/handouts/S1.../SLIDE-PE-1.ppt ………….. 24/8/2012
METODE ANALISIS • 3. Analysis methods used when reduced to nominal level of agree vs. disagree • Chi –square test • Cochran Q test • McNemar test AnalisisKhi- Kuadrat Analisis chi-square yang akandigunakanuntukmencariapakahadahubungan (asosiasi) antarvariabel-variabelkategoriktersebut Analisis chi-square didasarkanpadatabelkontingensi (seringiugadisebuttabulasisilang). Tabelkontingensiadalahtabel yang sel-selnyaberisifrekuensidariperpotonganbarisdankolom. Bentukumumdaritabelkontingensidenganvariabelpertamamemiliki m kategoridanvariabelkeduamemiliki k kategori , sebagaiberikut. Tabelkontingensiduaarah: Diunduhdari: http://digensia.wordpress.com/2012/03/26/koefisien-korelasi-cramer-c/………….. 24/8/2012
KEUNTUNGAN SEKALA LIKERT • Item analysis increases the degree of homogeneity or internal consistency in the set of statements. • Subjects generally find it easy to respond because they have a wide range of answers(usually five) to choose from instead of only two alternative responses, i.e., agree or disagree. • No outside group of judges is involved in selecting statements and giving values to them. KETERBATASAN SEKALA LIKERT • Ties in ranks occur quite frequently. • The response pattern of an individual is not revealed. • A respondent is required to answer all questions on the scale. • A problem of interpretation arises with this type of scale. • In this scale all statements of a universe are deemed to be of equal attitude value. Diunduhdari: ………….. 23/8/2012
Likert Scales • A very popular rating scale • Measures the feelings/degree of agreement of the respondents • Ideally, 4 to 7 points • Examples of 5-point surveys • Agreement SD D ND/NA A SA • Satisfaction SD D ND/NS S SS • Quality VP P Average G VG A typical Likert scale looks like this: Diunduhdari: http://rmsbunkerblog.wordpress.com/2010/09/22/rms-scale-week-2010-likert-scale-%E2%80%93-market-research-in-syracuse-ny-upstate-central-new-york-survey-focus-group/ ………….. 24/8/2012
Likert Scale The Likert scale requires the respondents to indicate a degree of agreement or disagreement with each of a series of statements about the stimulus objects. Strongly Disagree Neither Agree Strongly disagree agree nor agree disagree 1. Sears sells high quality merchandise. 1 2X 3 4 5 2. Sears has poor in-store service. 1 2X 3 4 5 3. I like to shop at Sears. 1 2 3X 4 5 • The analysis can be conducted on an item-by-item basis (profile analysis), or a total (summated) score can be calculated. • When arriving at a total score, the categories assigned to the negative statements by the respondents should be scored by reversing the scale. Using Attitudinal Data to Identify Latent Classes that Vary in Their Preference for Landscape Preservation Edward Morey, Mara Thiene, Maria De Salvo and Giovanni Signorello Forthcoming in Ecological Economics 2008 or 2009 The likelihood of significant heterogeneity in preferences for landscape preservation should be accounted for when designing WTP questions, estimating WTP, and formulating resulting policy recommendations. Herein, heterogeneity in preferences for landscape preservation is investigated in the context of a latent-class model under the assumption of the existence of some finite number of preference classes/groups. The number of classes is estimated, so few restrictions are placed on the form of the heterogeneity. One estimates the probability that individual i belongs to class c where these probabilities are a function of observable characteristics of the individual (covariates); this is much more flexible than assuming, for example, that all farmers have the same preferences. This paper aims to identify preference classes for landscape preservation in the IBLEO, a rural and beautiful part of Sicily. Estimation of classes is performed using only attitudinal data consisting of answers to Likert-scale questions about the importance of preservation and why the respondent thinks preservation is, or is not, important. Summarizing the results, estimation indicates four distinct preference classes. The classes vary in the level of importance attached to preservation and the motivation for preservation (e.g. use vs. non-use motivations), and include one group that has little interest in preservation. Diunduhdari: http://www.colorado.edu/economics/morey/papers/MoreyThieneDeSalvoSignorelloEcolEconomics.pdf ………….. 25/8/2012
Likert Scales: Advantages (summated rating = real name)RensisLikert, 1903–1981 • Easy for respondents to complete, most people familiar with the scale • Relatively easy to construct • Most popular attitudinal measure • Easy to score and analyze • Each item considered to be of equal attitude value (weight) -- homogeneous items Karin Braunsberger, Roger Gates, (2009) Developing inventories for satisfaction and Likert scales in a service environment Journal of Services Marketing, Vol. 23 Iss: 4, pp.219 – 225. The purpose of this paper is to produce up-to-date inventories for satisfaction and Likert scales that contain commonly used scale point descriptors and their respective mean scale values and standard deviations. All data were collected online using the SSI Survey Spot Panel. This panel is national (USA) in scope. Thirty-nine satisfaction items and 19 agreement items were tested on a random sample consisting of individuals 21-65 years old. The mean value and the standard deviation were calculated for each of these descriptors. Even though only six of the items that had been tested by Jones and Thurstone (1955) were included in the list of satisfaction scale descriptors, the semantic meanings of those six have changed very little over the years. One limitation might be that scale point descriptor inventories developed within the context of health insurance might not be valid in other service contexts. Since the present study focuses on Likert and satisfaction scales which are frequently used in service environments, the major contribution of this study is to provide services marketers with quantitative measurement of the meanings of commonly used scale point descriptors. This permits the development of successive and/or equal interval scales and thus aids in the analyses of data sets. It will thus help service marketers to develop questionnaires that more accurately reflect actual consumer satisfaction and opinions. Diunduhdari: http://www.emeraldinsight.com/journals.htm?articleid=1800605………….. 24/8/2012
Likert Scale Construction • Identify the attitudinal object and delimit it quite specifically. • Compose a series of statements about the attitudinal object that are half positive and half negative and are not extreme, ambiguous, or neutral. • Establish (a minimum of ) content validity with the help of an expert panel. • Pilot test the statements to establish reliability (Cronbach’s alpha) for each domain. • Eliminate statements that negatively affect internal consistency. • Construct the final scale by using the fewest number of items while still maintaining validity and reliability; create a balance of positive and negative items [Remember to reverse-code when summing]. • Administer the scale and instruct respondents to indicate their level of agreement with each statement. • Sum each respondent’s item scores to determine attitude. Diunduhdari: ………….. 23/8/2012
Likert Scale Instrument Construction Indoor air quality : perception versus reality in a rural hospital setting Goddard, Dianna Marie The objective of this study was to evaluate occupant perception of indoor air quality in a rural Arkansas hospital. Three generally accepted standard parameters of indoor air quality were examined---carbon dioxide levels, temperature, and relative humidity---for comparison to recommended standards of these values. A review of the literature revealed a lack of information in the subject of indoor air quality concerned with perception. Experimental data was obtained using two real-time monitoring instruments that logged work environment levels of carbon dioxide, carbon monoxide, relative humidity, and temperature. The results were tabulated and graphically formatted for ease of interpretation. In addition, an occupant survey containing a Likert scale was also used to determine the predictability of indoor air quality based on the individual responses from the surveys. Collectively, the data does not provide any conclusive evidence that occupant perception is a valid indicator of actual indoor air quality.Further investigation in this subject area of indoor air quality is needed. A better understanding of how air quality perception relates to actual indoor air quality will help to simplify the challenges that face air quality practitioners. • Use the general criteria for attitude statements. • Begin with non-threatening, easy items first; demographics last. • Have clear instructions with an example. • Anticipate data entry and analysis. • Anticipate missing data on items. • Use approved layout techniques. Diunduhdari: http://sunzi.lib.hku.hk/ER/detail/hkul/4082419 ………….. 24/8/2012
Scaling of Statements Response scales vary. Recommend to use an even number of response categories (no neutral category) and a N/A response for agreement scales Label all response categories. Since this is a summated rating scale, the scale of measurement of the sum or mean is interval. Never analyze by item. Scale of measurement of any one item is ordinal. Anchored scales: frequency, importance, etc. (Odd # = OK) Pictures, thermometers, etc., may be used as scales Multiple scales per item may be used. Greater range in the scales produce more variability in the data: 8 better than 6, 6 better than 4, etc. (Correlations work better.) Likert scales may be subject to distortion from several causes. Respondents may avoid using extreme response categories (central tendency bias); agree with statements as presented (acquiescence bias); or try to portray themselves or their organization in a more favorable light (social desirability bias). Designing a scale with balanced keying (an equal number of positive and negative statements) can obviate the problem of acquiescence bias, since acquiescence on positively keyed items will balance acquiescence on negatively keyed items, but central tendency and social desirability are somewhat more problematic. Diunduhdari: http://en.wikipedia.org/wiki/Likert_scale Diunduhdari: ………….. 23/8/2012
Likert Scaling • Even Number of Response Categories • Label all categories • Use N/A if appropriate [No neutral/undecided] • Frequency, Importance, etc. [Anchored] Likert scaling is a bipolar scaling method, measuring either positive or negative response to a statement. Sometimes an even-point scale is used, where the middle option of "Neither agree nor disagree" is not available. This is sometimes called a "forced choice" method, since the neutral option is removed. The neutral option can be seen as an easy option to take when a respondent is unsure, and so whether it is a true neutral option is questionable. It has been shown that when comparing between a 4-point and a 5-point Likert scale, where the former has the neutral option unavailable, the overall difference in the response is negligible. Diunduhdari: http://en.wikipedia.org/wiki/Likert_scale Diunduhdari: ………….. 23/8/2012 Diunduhdari: ………….. 23/8/2012
Summative Ratings Community Perceptions toward Economic and Environmental Impacts of Tourism on Local Communities FariborzAref , Ma’rofRedzuan and Sarjit S. Gill Asian Social Science. July, 2009. Vol. 5, No. 7 This paper investigates the community perceptions toward economic and environmental impacts of tourism in Shiraz, Iran. Special focus is on the differences in perceptions between the Old and New Districts of Shiraz. The study demonstrates that there are broadly similar views among the community leaders and community residents from both districts of Shiraz. In fact, a high percentage of the answers obtained highlighted positive aspects environmental and economic impacts of tourism toward local communities. According to the survey, the strongest and favourable perceptions toward tourism impacts are found to be linked with environmental aspects and while economic matters are found to be the least favourable in terms of the perceived impacts on tourism. T-test analysis of the study indicates that there is no significant difference between community leaders' perceptions in both districts of Shiraz City. Results drew from discussion with the target group show that the community residents have positive perceptions toward economic and environmental impacts of tourism with only minor differences with each other. The questionnaire was structured around a Likert scale. The items for community perceptions toward tourism impacts were taken from these studies. The respondents answered to each statement based on five scales. The value of each response for these items on the questionnaire is as follows: 1 = strongly disagree 2 = disagree 3 = not sure 4 = agree 5 = strongly agree. Ko & Stewart (2002) and Maddox (1985), recommended the use of a Likert type scale in tourism research due to its high validity. Then, the questionnaire was piloted tested to have its content validated by several reviewers of Persian background. Statements for tourism impacts were tested for their validity using Cronbach’s alpha. The participants in the pilot test had relatively diverse demographic characteristics, especially with regards to community. The t-test was employed to test to determine whether there were significant differences among group mean totals and item mean scores. Means and standard deviations are the descriptive statistics used in discussing the distribution of responses gathered during the quantitative component of this study. To assess the normality of the distribution of the data, the skewness and kurtosis of each variable were also examined. According to George & Mallery (2002) if the coefficient of the skewness and kurtosis falls between -0.5 and +0.5 inclusive, then the distribution appears to be relatively symmetric which in this study skewness was .254 and Kurtosis -.211. • A number of items collectively measure one construct (Job Satisfaction) • A number of items collectively measure a dimension of a construct and a collection of dimensions will measure the construct (Self-esteem) Diunduhdari: journal.ccsenet.org/index.php/ass/.../2746………….. 24/8/2012
Summative Likert Scales J Air Waste Manag Assoc. 2000 Jul ;50 (7):1081-94. Exposure of chronic obstructive pulmonary disease patients to particulate matter: relationships between personal and ambient air concentrations. S. T. Ebelt, A J Petkau, S Vedal, T V Fisher, M Brauer. Mot time-series studies of particulate air pollution and acute health outcomes assess exposure of the study population using fixed-site outdoor measurements. To address the issue of exposure misclassification, we evaluate the relationship between ambient particle concentrations and personal exposures of a population expected to be at risk of particle health effects. Sampling was conducted within the Vancouver metropolitan area during April-September 1998. Sixteen subjects (non-smoking, ages 54-86) with physician-diagnosed chronic obstructive pulmonary disease (COPD) wore personal PM2.5 monitors for seven 24-hr periods, randomly spaced approximately 1.5 weeks apart. Time-activity logs and dwelling characteristics data were also obtained for each subject. Daily 24-hr ambient PM10 and PM2.5 concentrations were measured at five fixed sites spaced throughout the study region. SO4(2-), which is found almost exclusively in the fine particle fraction and which does not have major indoor sources, was measured in all PM2.5 samples as an indicator of accumulation mode particulate matter of ambient origin. The mean personal and ambient PM2.5 concentrations were 18 micrograms/m3 and 11 micrograms/m3, respectively. In analyses relating personal and ambient measurements, ambient concentrations were expressed either as an average of the values obtained from five ambient monitoring sites for each day of personal sampling, or as the concentration obtained at the ambient site closest to each subject's home. The mean personal to ambient concentration ratio of all samples was 1.75 (range = 0.24 to 10.60) for PM2.5, and 0.75 (range = 0.09 to 1.42) for SO4(2-). Regression analyses were conducted for each subject separately and on pooled data. The median correlation (Pearson's r) between personal and average ambient PM2.5 concentrations was 0.48 (range =-0.68 to 0.83). Using SO4(2-) as the exposure metric, the median r between personal and average ambient concentrations was 0.96 (range = 0.66 to 1.0). Use of the closest ambient site did not improve the median correlation of the group for either PM2.5 or SO4(2-). All pooled analyses resulted in lower correlation coefficients than the median correlation coefficient of individual regressions. Personal SO4(2-) was more highly correlated with all ambient measures than PM2.5. Inclusion of time-activity and dwelling characteristics data did not result in a useful predictive regression model for PM2.5 personal exposure, but improved the model fit from simply regressing against ambient concentration (R2 = 0.27). The model for SO4(2-) was predictive (R2 = 0.82), as personal exposures were largely explained by ambient levels. These results indicate a relatively low correlation between personal exposure and ambient PM2.5 that is not improved by assigning exposure to the closest ambient monitor. The correlation between personal exposure and ambient concentration is high, however, when using SO4(2-), an indicator of accumulation mode particulate matter of ambient origin. • Must contain multiple items • Each individual item must measure something that has an underlying, quantitative measurement continuum • There can be no right/wrong answers as opposed to multiple-choice questions • Items must be statements to which the respondent assigns a rating • Cannot be used to measure knowledge or ability, but familiarity Diunduhdari: http://lib.bioinfo.pl/paper:10939202 ………….. 24/8/2012
SCALE CONSTRUCTION • Define Constructs • Conceptual/theoretical basis from the literature • Are their sub-scales (dimensions) to the scale • Multiple item sub-scales • Principle of Parsimony (kesederhanaan, kehematan) • Simplest explanation among a number of equally valid explanations must be used. DATA VARIABEL LATENT (KONSTRUK) Penelitiandibidangekoilogi-ekonomidanseringmelibatkanvariabel yang tidakdapatdiukursecaralangsung, disebutvariabel latent atau unobservable; misalnyakepuasan, motivasidanlainnya. Pengukuranvariabellatenmenggunakaninstrumenberupakuisionerakanmenghasilkan data darisetiapindikatoratau data darisetiap item. Olehkarenaitu, indikatoratau item seringdisamakandenganvariabel manifest atauvariabel observable. Untukmemperoleh data darivariabel latent atauvariabel unobservable dapatdilakukandenganbeberapacara, antara lain: 1. Metode Total Skor 2. Metode Rata-Rata Skor 3. Metode Rescoring 4. MetodeIndikatorTerkuat 5. MetodeSkorFaktor 6. MetodeSkorKomponenUtama Metodepertamaberartimenjumlahkanskorsemuaindikator, shinggadiperoleh data total skor yang merupakan data variabellatenbersangkutan. Sedangkanmetodekeduamenggunakan rata-rata skorindikator. Sebagaiilustrasidigunakan data rekaandibawahini (menggunakanskalaLikert 1 sampai 5). Diunduhdari: http://anaarisanti.blogspot.com/2010/05/data-variabel-latent.html………….. 25/8/2012
DATA VARIABEL LATENT (KONSTRUK) Metode Rescoring Metodeinimerubah total skormenjadiskalaawal (1 sampai 5). Caranyaadalah, untuk data diatas, sebagaiberikut: - Nilai minimal skor total yang mungkinadalah 3 - Nilaimaksimalskor total yang mungkinadalah 15 - Range = 15 – 3 = 12 - Interval kelas (banyaknyaskorawal, 1 sampai 5) adalah 5 - Lebar interval kelas = 12/5 = 2.4 Rescoring bernilai 1 jikanilaiskor total antara 3 sampai (3 + 2.4) = 5.4 Rescoring bernilai 2 jikanilaiskor total antara >5.4 sampai (5.4 + 2.4) = 7.8 Rescoring bernilai 3 jikanilaiskor total antara >7.8 sampai 10.2 Rescoring bernilai 4 jikanilaiskor total antara >10.2 sampai 12.6 Rescoring bernilai 5 jikanilaiskor total antara >12.6 sampai 15 Untukobservasipertama, nilaiskor total adalah 7, dimana 7 beradapadaselang rescoring 2, yaitu >5.4 sampai 7.8. Demikianseterusnya. Ketigametodeinibersifatsetiapindikatordipandangmemilikibobot yang sama, daninformasi 100% terpakaiatautercakupdalamvariabel latent. Diunduhdari: http://anaarisanti.blogspot.com/2010/05/data-variabel-latent.html………….. 25/8/2012
METODE INDIKATOR TERKUAT MetodeIndikatorTerkuatinimenggunakanindikatorterkuat. Indikatorterkuatdiperolehdarihasilkorelasiantarmasing-masingindikatordengan total skor. Indikator yang memilikikorelasiterbesardipandangsebagaiindikatorterkuatdandigunakanuntukmewakilivariabel latent. Nilaikorelasiantarasetiapindikatordengan total skor: Dari hasilanalisistersebutdiperolehnilaikorelasiantaraindikator 1 denganskor total adalah 0.625, indikator 2 denganskor total adalah 0.832 danindikator 3 denganskor total adalah 0.790. Indikator yang memilikikorelasitertinggiadalahindikator 2 (0.832), sehinggavariabel latent yang digunakanmenggunakanskorindikator 2. Diunduhdari: http://anaarisanti.blogspot.com/2010/05/data-variabel-latent.html………….. 25/8/2012
METODE SKOR FAKTOR & METODE SKOR KOMPONEN UTAMA KeduaMetodeinimenggunakananalisisfaktordananalisiskomponenutama. Metodeinimenghasilkanskorfaktordanskorkomponenutama, yang dijadikansebagai data untukvariabel latent. Keduametodeiniberbedadenganketigametodepertamayaitubobotmasing-masingindikatoradalahberbeda, dantidak 100% informasiterpakaiatautercakup. Keduametodeterakhiriniakandijelaskanpada sub babtersendiripadababini. Perbedaanmasing-masingmetodedapatdilihatdarigambarberikut: Padakeduagambartampakterlihatperbedaanterletakbagaimanaarahhubunganantaravariabellatendenganindikator. Padaanalisisfaktor, masing-masingvariabelindikatoradalahfungsidarivariabel latent, sedangkanpadaanalisiskomponenutama, variabel latent adalahfungsidariseluruhvariabelindikator. Konstrukdengananalisisfaktormenganggapbahwavariabel latent adalahrefleksidarisejumlahindikator, sedangkankonstrukdengananalisiskomponenutamamenganggapbahwavariabel latent dibentuk (formasi) darisejumlahindikator. Olehkarenaitu, pembentukanvariabel latent menggunakananalisisfaktordinamakanbentukreflektif, sedangkanpembentukanvariabel latent menggunakananalisiskomponenutamadinamakanbentukformatif. Diunduhdari: http://anaarisanti.blogspot.com/2010/05/data-variabel-latent.html………….. 25/8/2012
ITEM CONSTRUCTION • Agreement items • Write declarative statements • Death penalty should be abolished • I like to listen to classical music • Frequency items (how often) • I like to read • Evaluation items • How well did your team play • How well does the police serve your community ProsedurdalammembuatskalaLikertadalahsebagaiberikut: Penelitimengumpulkanbahan-bahan yang relevant denganmasalah yang sedangditeliti Menyusun Blue Printuntukmemandupenyusunanalatukur Membuat item-item yang akandiujisesuaidenganpanduanUjicoba itemkepadasekelompokresponden yang cukuprepresentatifdaripopulasi yang inginditeliti. Respondendiatasdimintauntukmengecektiap item, apakahiamenyenangi (+) atautidakmenyukainya (-). Responstersebutdikumpulkandanjawaban yang memberikanindikasimenyenangidiberiskortertinggi. Tidakadamasalahuntukmemberikanangka 5 untuk yang tertinggidanskor 1 untuk yang terendahatausebaliknya. Yang pentingadalahkonsistensidariarahsikap yang diperlihatkan. Demikianjugaapakahjawaban “setuju” atau “tidaksetuju” disebut yang disenangi, tergantungdariisipertanyaandanisidari item-item yang disusun. Setelah item diujicobakepadaresponden, laludiujitingkatvaliditasdanreabilitasdari item-item tersebut. Validitasadalahsuatuukuran yang menunjukkantingkatankevalidanataukesahihansuatuinstrumensedangkanreliabilitasmerupakanpenilaiantingkatkonsistensiterhadaphasilpengukuranbiladilakukanmultiple measurement padasebuahvariabelsuatualatukurdikatakanreliabeljikaalatukurtidakberubah. Diunduhdari: http://kiptykipty.wordpress.com/2010/06/05/skala-likert-dalam-teknik-evaluasi-perencanaan/ ………….. 25/8/2012
PENULISAN ITEM • Mutually exclusive and collectively exhaustive items • Use positively and negatively phrased questions • Avoid colloquialism, expressions and jargon • Avoid the use of negatives to reverse the wording of an item • Don’t use: I am not satisfied with my job • Use: I hate my job! • Be brief, focused, and clear • Use simple, unbiased questions Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM Instrumenataualatpengumpul data adalahalat yang digunakanuntukmengumpulkan data dalamsuatupenelitian. Data yang terkumpuldenganmenggunakaninstrumentertentuakandideskripsikandandilampirkanataudigunakanuntukmengujihipotesis yang diajukandalamsuatupenelitian. AlurPenyusunandanPengembanganInstrumen: Variabel Teori Konstruk Definisi Konseptual Definisi Operasional PenetapanJenis Instrumen MenyusunButir Instrumen Berdasarkansintesisdariteori-teori yang dikajitentangsuatukonsepdarivariabel yang hendakdiukur, kemudiandirumuskankonstrukdarivariabeltersebut. Konstrukpadadasarnyaadalahbangunpengertiandarisuatukonsep yang dirumuskanolehpeneliti. Adabeberapajenisinstrumen yang biasadigunakandalampenelitian, antara lain kuesioner, skala (skalasikapatauskalapenilaian), tes, dan lain-lain. Kuesioneradalahalatpengumpul data yang berbentukpertanyaan yang akandiisiataudijawabolehresponden. Beberapaalasandigunakannyakuesioneradalah : kuesionerterutamadipakaiuntukmengukurvariabel yang bersifatfaktual, untukmemperolehinformasi yang relevandengantujuanpenelitian, dan Untukmemperolehinformasidenganvaliditasdanreliabilitassetinggimungkin. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM (BUTIR –BUTIR TES) TipePilihanGanda Item hendaklahmenanyakanhal yang pentinguntukdiketahui. Tulislah item yang berisipernyataanpasti. Utamakan item yang mengandungpernyataanumum yang bertahan lama. Buatlah item yang berisihanyasatugagasansaja. Buatlah item yang menyatakanintipertanyaandenganjelas. kalimatsederhanadantidakberlebih-lebihan. Sebaiknya item tidakdidasariolehpernyataannegatif. Gunakanbahasa yang jelas, kata yang sederhana, danpernyataan yang langsung. Item harusmemberikanalternatifbagiisipernyataan yang paling penting. Berikanalternatifjawaban yang jelasberbeda. Alternatif yang ditawarkanhendaknyamempunyaistrukturdanarti yang sejajarataudalamsatukategori. Penggunaanalternatif yang semata-matameniadakanataubertentangandenganalternatif yang lain, haruslahdihindari. Bilamanamungkin, susunlahalternatifjawabandalamurutanbesarnyaatauurutanlogisnya. Penggunaanalternatif “bukansalah-satudiatas” atau “semua yang diatas” hanyabaikapabilakebenaranbersifatmutlakdanbukansemata-matamasalahlebihdankurangbaikataumasalahkebenaranrelatif. Janganmenjebaksiswadenganmenanyakanhal yang tidakadajawabannya. Hindaripenggunaankata-kata yang dapatdijadikanpetunjukolehsiswadalammenjawab. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM (BUTIR –BUTIR TES) TipeBenar-Salah Kaidahataupetunjukpenulisan item tipebenar–salahtelahdikemukakanolehEbel (1979) sebagaimanaberikutini. Item haruslahmengungkapideataugagasan yang penting. Item tipebenar-salahhendaknyamengujipemahaman, mengungkapingatanmengenaisuatufaktaatauhafalan. Kebenaranatauketidakbenaransuatu item haruslahbersifatmutlak. Item harusmengujipengetahuan yang spesifikdanjawabannyatidakjelasbagisemuaorang, kecualibagimereka yang menguasaipelajaran. Item harusdinyatakansecarajelas. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM (BUTIR –BUTIR TES) TipeJawabanPendek Pernyataanataupertanyaan item harusditulisdenganhati-hatisehinggadapatdijawabdenganhanyasatujawaban yang pasti. Sebaiknyarumuskanjawabannyalebihdahulubarukemudianmenulispertanyaannya. Gunakanpertanyaanlangsung, kecualibilamana model kalimattakselesaiakanmemungkinkanjawaban yang lebihjelas. Usahakan agar dalampertanyaantidakterdapatpetunjuk yang mungkindigunakanolehsubjekdalammenjawab item. Janganmenggunakankataataukalimat yang langsungdikutipdaribuku. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM (BUTIR –BUTIR TES) TipeKarangan (Esai) Berikanpertanyaanatautugas yang mengarahkanpenjawabpertanyaan (siswa) agar dapatmenunjukkanpenguasaanpengetahuan yang penting. Buatlahpertanyaan yang arahjawabannyajelas, sehinggaparaahlidapatsetujubahwasatujawabanakanlebihbaikdaripada yang lainnya. Janganmenanyakansikapataupendapat. Sebaiknyapertanyaandiawaliolehkata-kataseperti, “Bandingkan …”, “Berikanalasan …”, “Jelaskanmengapa …”, “Bericontoh …”, dansemacamnya. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM (BUTIR –BUTIR TES) TipePasangan Premisdanresponshendaknyadibuatdalamjumlah yang tidaksama. Baikpremismaupunresponsharuslahberisihal yang homogen, yaitudarisejeniskategoriisi. Usahakan agar premisdanresponsnyaberisikalimat-kalimatataukata yang pendek. Buatlahpetunjukpemasangan yang jelas, sehinggapenjawabsoalataupertanyaanmengetahuidasarapakah yang harusdigunakandalammemasangkanpremisdanresponsnya. Sedapatmungkinsusunlahpremisdanresponsmasing-masingsecaraalfabetikataumenurutbesarankuantitatifnya. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM UNTUK SKALA LIKERT Untukmenulispernyataansikap yang bermutu, penyusunskalaharusmenurutisuatukaidahataupedomanpenulisanpernyataan agar ciri-ciripernyataansikaptidakterlupakandan agar setiappernyataanmempunyaikemampuanmembedakanantarakelompokresponden yang setujudengankelompokresponden yang tidaksetujuterhadapobjeksikap. BeberapapetunjukuntukmenyusunskalaLikertdiantaranya : Tentukanobjek yang dituju, kemudiantetapkanvariabel yang akandiukurdenganskalatersebut. Lakukananalisisvariabeltersebutmenjadibeberapa sub variabelataudimensivariabel, lalukembangkanindikatorsetiapdimensitersebut. Dari setiapindikatordiatas, tentukanruanglingkuppernyataansikap yang berkenaandenganaspekkognisi, afeksi, dankonasiterhadapobjeksikap. Susunlahpernyataanuntukmasing-masingaspektersebutdalamduakategori, yaknipernyataanpositifdanpernyataannegatif, secaraseimbangbanyaknya. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN ITEM UNTUK SKALA LIKERT Edwards (1957) meramuberbagai saran danpetunjukdariparaahlimenjadisuatupedomanataukriteriapenulisanpernyataansikap. Beberapakriteria yang dimaksudadalahsebagaiberikut. Janganmenulispernyataan yang membicarakanmengenaikejadian yang telahlewatkecualikalauobjeksikapnyaberkaitandenganmasalalu. Janganmenulispernyataan yang berupafaktaataudapatditafsirkansebagaifakta. Janganmenulispernyataan yang dapatmenimbulkanlebihdarisatupenfsiran. Janganmenulispernyataan yang tidakrelevandenganobjekpsikologisnya. Janganmenulispernyataan yang sangatbesarkemungkinannyaakandisetujuiolehhampirsemuaorangataubahkanhampirtakseorang pun yang akanmenyetujuinya. Pilihlahpernyataan-pernyataan yang diperkirakanakanmencakupkeseluruhanliputanskalaafektif yang diinginkan. Usahakan agar setiappernyataanditulisdalambahasa yang sederhana, jelas, danlangsung. Janganmenuliskanpernyataandenganmenggunakankalimat- kalimat yang rumit. Setiappernyataanhendaknyaditulisringkasdenganmenghindarikata-kata yang tidakdiperlukandan yang tidakakanmemperjelasisipernyataan. Setiappernyataanharusberisihanyasatuide (gagasan) yang lengkap. Pernyataan yang berisiunsur universal seperti “tidakpernah”, “semuanya”, “selalu”, “takseorang pun”, dansemacamnya, seringkalimenimbulkanpenafsiran yang berbeda-bedadankarenanyasedapatmungkinhendaklahdihindari. Kata-kataseperti “hanya”, “sekedar”, “semata-mata”, dansemacamnyaharusdigunakanseperlunyauntukmenghindarikesalahanpenafsiranisipernyataan. Janganmenggunakankataatauistilah yang mungkintidakdapatdimengertiolehpararesponden. Hindarilahpernyataan yang berisikatanegatifganda. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/
PENULISAN BUTIR UNTUK KUESIONER Cara menyusunkuesionerbesertabutir-butir yang tercantumdidalamnyaharuslahtetapmengacupadapedomanpenyusunaninstrumensecaraumum, sehinggaberlaku pula langkah-langkahsebagaimanatelahdijelaskandibagianterdahulu. Dimulaidengananalisisvariabel, pembuatankisi-kisi, dankemudiansampaipadapenyusunanpertanyaanuntukkuesioner. Secaralebihteknis, petunjukuntukmembuatkuesioneradalahsebagaiberikut. Mulaidenganpengantar yang isinyaberupapermohonanmengisikuesionersambilmenjelaskanmaksuddantujuannya. Jelaskanpetunjukataucaramengisinyasupayatidaksalah. berikancontohpengisiannya. Mulaidenganpertanyaanuntukmengungkapkanidentitasresponden. Dalamidentitasinisebaiknyatidakdimintamengisinama. Identitascukupmengungkapkanjeniskelamin, usia, pendidikan, pekerjaan, pengalaman, dan lain-lain yang adakaitannyadengantujuankuesioner. Isipertanyaansebaiknyadibuatbeberapakategoriataubagiansesuaidenganvariabel yang diungkapkan, sehinggamudahmengolahnya. Rumusanpertanyaandibuatsingkat, tetapimembingungkandanmenimbulkansalahpenafsiran. Hubunganantarapertanyaan yang satudenganpertanyaanlainnyaharusdijagasehinggatampakketerkaitanlogikanyadalamsaturangkaian yang sistematis. Hindaripenggolonganpertanyaanterhadapindikatorataupersoalan yang sama. Usahakan agar jawaban, yaknikalimatataurumusannyatidaklebihpanjangdaripadapertanyaan. Kuesioner yang terlalubanyakatauterlalupanjangakanmelelahkandanmembosankanrespondensehinggapengisiannyatidakobjektiflagi. Adabaiknyakuesionerdiakhiridengantandatangansipengisiuntukmenjaminkeabsahanjawabannya. Untukmelihatvaliditasjawabankuesioner, adabaiknyakuesionerdiberikankepadabeberaparespondensecaraacakdandilakukanwawancaradenganpertanyaan yang identikdenganisikuesioner yang telahdiisinya. Sumber: Dr.Ir. PudjiMuljono, Msi. DisampaikanpadaLokakaryaPeningkatanSuasanaAkademik JurusanEkonomi FIS-UNJ tanggal 5 sampaidengan 9 Agustus 2002 Diunduhdari: https://docs.google.com/viewer?a=v&q=cache:k1SsN7H88fAJ:repository.ipb.ac.id/bitstream/handle/