請教squid, jpg,flv,命中率問題
在學校內起了個代理伺服器,運行了一段時間,用的人還不多。
總體感覺不太好。
特發來相關日誌,請各位指點一下,如何改進。
目前不明白的問題
1,為什麼jpg的文件hit這麼低
2,對flv的文件,所看到的視頻數據,是FLV通過代理伺服器去下載,還是其新建連接繞過代理伺服器去下載。我希望其是繞過代理。
3,在學校起代理伺服器,通常命中率能到多少,現在的才到7%左右。
bash-2.05b# grep -v '^#' /data/app/squid/etc/squid.conf | sed '/^$/d'
acl all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl localnet src 127.0.0.1/32 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl SSL_ports port 443
acl Safe_ports port 80-89 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 1025-65535 # unregistered ports
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access deny all
icp_access allow localnet
icp_access deny all
http_port 3128
hierarchy_stoplist cgi-bin ?
cache_mem 800 MB
maximum_object_size_in_memory 64 KB
memory_replacement_policy lru
cache_dir ufs /data/cache 60000 64 256
maximum_object_size 524288 KB
logformat squid %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
logformat common %>a %tl %Hs %<st %rm %ru
access_log /data/app/squid/var/logs/access.log squid
logfile_rotate 100
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
range_offset_limit 200 KB
acl shoutcast rep_header X-HTTP09-First-Line ^ICY.
upgrade_http0.9 deny shoutcast
cache_vary on
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
coredump_dir /data/app/squid/var/cache
bash-2.05b# uname -a
FreeBSD cache 4.10-RELEASE FreeBSD 4.10-RELEASE #0: Tue Apr 25 14:16:49 CST 2006 wm@ns.com:/usr/src/sys/compile/cache i386
bash-2.05b# /data/app/squid/bin/squidclient mgr:info
HTTP/1.0 200 OK
Server: squid/2.7.STABLE7
Date: Wed, 06 Jan 2010 04:03:56 GMT
Content-Type: text/plain
Expires: Wed, 06 Jan 2010 04:03:56 GMT
X-Cache: MISS from xxxx
Via: 1.0 xxxx:3128 (squid/2.7.STABLE7)
Connection: close
Squid Object Cache: Version 2.7.STABLE7
Start Time: Tue, 05 Jan 2010 14:53:22 GMT
Current Time: Wed, 06 Jan 2010 04:03:56 GMT
Connection information for squid:
Number of clients accessing cache: 40
Number of HTTP requests received: 54288
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 68.7
Average ICP messages per minute since start: 0.0
Select loop called: 1603928 times, 29.573 ms avg
Cache information for squid:
Request Hit Ratios: 5min: 24.4%, 60min: 35.1%
Byte Hit Ratios: 5min: 1.0%, 60min: 34.6%
Request Memory Hit Ratios: 5min: 10.7%, 60min: 8.8%
Request Disk Hit Ratios: 5min: 27.4%, 60min: 54.3%
Storage Swap size: 16580580 KB
Storage Mem size: 145020 KB
Mean Object Size: 22.77 KB
Requests given to unlinkd: 5769
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.42149 0.39928
Cache Misses: 0.76407 0.76407
Cache Hits: 0.01035 0.00767
Near Hits: 0.37825 0.42149
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.00190 0.00190
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 47433.481 seconds
CPU Time: 117.092 seconds
CPU Usage: 0.25%
CPU Usage, 5 minute avg: 0.38%
CPU Usage, 60 minute avg: 0.32%
Process Data Segment Size via sbrk(): 396112 KB
Maximum Resident Size: 398944 KB
Page faults with physical i/o: 3
Memory accounted for:
Total accounted: 214134 KB
memPoolAlloc calls: 11951234
memPoolFree calls: 9398207
File descriptor usage for squid:
Maximum number of file descriptors: 32768
Largest file desc currently in use: 106
Number of file desc currently in use: 30
Files queued for open: 0
Available number of file descriptors: 32738
Reserved number of file descriptors: 100
Store Disk files open: 0
IO loop method: kqueue
Internal Data Structures:
728839 StoreEntries
14630 StoreEntries with MemObjects
14625 Hot Object Cache Items
728155 on-disk objects
《解決方案》
bash-2.05b# ./scalar.awk /data/app/squid/var/logs/access.log.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~ SCALAR (Squid Cache Advanced Log Analyzer & Reporter), Version 0.96 ~~~~
~~~~ ~~~~
~~~~ (C) 2003-4 by Yuri N. Fominov, YuriF@risk.az, http://scalar.risk.az ~~~~
~~~~ SCALAR has no warranty and it is completely free, so you are welcome ~~~~
~~~~ to re-distribute this pretty useful piece of software. Good Luck2All ~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~ Analysis Headlines ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Log Start Time [-26-02-2010 01:01:07]
Log End Time [-25-02-2010 01:00:59]
Lines Analyzed 1.617M
Unique Clients: 50
In Traffic: 8.634 GB
Out Traffic: 9.347 GB
------------------------------------
Saved Traffic: 730.187 MB 7.63 %
~~~ Requests By Hours ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
289.0K ==
274.6K ==
260.1K ==
245.7K ==
231.2K == ==
216.8K == ==
202.3K == ==
187.8K == ==
173.4K == == ==
158.9K == == == ==
144.5K == == == ==
130.1K == == == == ==
115.6K == == == == ==
101.2K == == == == ==
86.7K == == == == == ==
72.2K == == == == == == == == ==
57.8K == == == == == == == == ==
43.4K == == == == == == == == == == == ==
28.9K == == == == == == == == == == == == == == ==
14.4K == == == == == == == == == == == == == == == ==
_______________________________________________________________________________
Hours: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Maximum Requests: 289.018K/hour, 4.816K/min, 80.3/sec.
Average Requests: 67.231K/hour, 1.121K/min, 18.7/sec.
Minimum Requests: 1 /hour, 0 /min, 0.0/sec.
~~~ Traffic by Hours ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1.6GB ==
1.5GB ==
1.4GB ==
1.3GB ==
1.3GB ==
1.2GB == ==
1.1GB == ==
1.0GB == ==
966.3MB == ==
885.8MB == ==
805.2MB == ==
724.7MB == == == ==
644.2MB == == == ==
563.7MB == == == == == ==
483.1MB == == == == == == == == == ==
402.6MB == == == == == == == == == == == ==
322.1MB == == == == == == == == == == == == == ==
241.6MB == == == == == == == == == == == == == ==
161.0MB == == == == == == == == == == == == == ==
80.5MB == == == == == == == == == == == == == == ==
_______________________________________________________________________________
Hours: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Maximum Xfers: 1.573 GB/hour, 26.841 MB/min, 458.094 KB/sec
Average Xfers: 398.339 MB/hour, 6.639 MB/min, 113.305 KB/sec [ 906 kbps]
Minimum Xfers: 802 B/hour, 13 B/min, 0 B/sec [ 0 kbps]
~~~ File Extensions Report ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-------|------ R E Q U E S T S --------|----------- T R A F F I C -----------|
Ext. | total | misses | hits |hit% | total | misses | hits |hit%|
-------|--------|--------|--------|-----|----------|----------|----------|----|
gif 202.206K 21.624K 180.582K 89% 213.145 MB 76.872 MB 136.273 MB 64%
<QUERY> 88.906K 80.989K 7.917K 9% 3.346 GB 3.228 GB 120.036 MB 4%
jpg 65.845K 46.594K 19.251K 29% 1.155 GB 1.008 GB 150.827 MB 13%
js 17.315K 3.772K 13.543K 78% 126.336 MB 21.499 MB 104.837 MB 83%
/ 17.124K 14.722K 2.402K 14% 138.026 MB 125.285 MB 12.741 MB 9%
png 8.997K 3.571K 5.426K 60% 65.503 MB 38.417 MB 27.086 MB 41%
css 6.078K 1.288K 4.790K 79% 37.640 MB 8.340 MB 29.300 MB 78%
swf 5.005K 1.068K 3.937K 79% 130.284 MB 29.743 MB 100.542 MB 77%
php 4.523K 4.429K 94 2% 8.845 MB 8.600 MB 250.667 KB 3%
xml 3.947K 1.126K 2.821K 71% 13.094 MB 7.283 MB 5.811 MB 44%
html 3.503K 1.916K 1.587K 45% 26.186 MB 22.508 MB 3.678 MB 14%
htm 2.443K 1.040K 1.403K 57% 22.181 MB 20.009 MB 2.172 MB 10%
<LONG.> 1.567K 1.566K 1 0% 14.679 MB 14.653 MB 26.418 KB 0%
cab 1.114K 1.113K 1 0% 129.268 MB 128.207 MB 1.062 MB 1%
do 1.003K 991 12 1% 14.537 MB 14.472 MB 67.134 KB 0%
ico 844 555 289 34% 1.856 MB 1.106 MB 767.402 KB 40%
mp3 466 139 327 70% 91.770 MB 67.403 MB 24.367 MB 27%
ini 371 174 197 53% 741.714 KB 318.693 KB 423.021 KB 57%
shtml 360 337 23 6% 9.971 MB 9.728 MB 248.812 KB 2%
jpeg 341 261 80 23% 10.213 MB 8.628 MB 1.585 MB 16%
wma 325 322 3 1% 10.136 MB 10.135 MB 943 B 0%
314 233 81 26% 14.376 MB 13.570 MB 824.768 KB 6%
rp 251 71 180 72% 3.330 MB 1.021 MB 2.309 MB 69%
flv 238 229 9 4% 1.191 GB 1.189 GB 1.799 MB 0%
aspx 208 207 1 0% 2.522 MB 2.522 MB 0 B 0%
cgi 179 179 0 0% 119.790 KB 119.790 KB 0 B 0%
json 175 31 144 82% 422.825 KB 207.613 KB 215.212 KB 51%
txt 164 123 41 25% 497.592 KB 451.632 KB 45.960 KB 9%
zip 143 47 96 67% 5.023 MB 4.562 MB 472.119 KB 9%
dll 137 137 0 0% 1.008 MB 1.008 MB 0 B 0%
: 136 136 0 0% 2.171 MB 2.171 MB 0 B 0%
bmp 105 92 13 12% 1.781 MB 1.432 MB 357.445 KB 20%
cur 101 23 78 77% 133.299 KB 25.776 KB 107.522 KB 81%
inf 101 27 74 73% 192.741 KB 44.479 KB 148.262 KB 77%
dat 85 84 1 1% 2.219 MB 2.219 MB 475 B 0%
vbs 83 0 83 100% 54.494 KB 0 B 54.494 KB 100%
jhtml 82 78 4 5% 1.190 MB 1.186 MB 3.773 KB 0%
asp 78 78 0 0% 345.646 KB 345.646 KB 0 B 0%
crl 70 35 35 50% 3.430 MB 3.419 MB 10.771 KB 0%
zip1 68 22 46 68% 1.174 MB 419.694 KB 782.280 KB 65%
ff 65 13 52 80% 742.178 KB 148.436 KB 593.742 KB 80%
jsp 53 49 4 8% 375.016 KB 373.891 KB 1.125 KB 0%
mp4 50 50 0 0% 1.002 GB 1.002 GB 0 B 0%
exe 41 39 2 5% 54.642 MB 54.499 MB 145.771 KB 0%
dwr 38 38 0 0% 75.236 KB 75.236 KB 0 B 0%
q 36 36 0 0% 13.279 KB 13.279 KB 0 B 0%
14 32 22 10 31% 232.991 KB 90.956 KB 142.035 KB 61%
conf 30 1 29 97% 107.807 KB 3.584 KB 104.223 KB 97%
ani 28 16 12 43% 156.200 KB 52.203 KB 103.997 KB 67%
cn:443 27 27 0 0% 866.708 KB 866.708 KB 0 B 0%
================|========|========|=====|==========|==========|==========|====|
TOTALS: 435.401K 189.720K 245.681K 56% 7.827 GB 7.114 GB 730.187 MB 9%
-----------------------|-- R E Q U E S T S ---|-------- T R A F F I C --------|
Content Type | total | hits |hit%| total | hits |hit%|%VOL|
-----------------------|--------|--------|----|----------|----------|----|----|
Images/Graphics & Flash 283.343K 209.578K 74% 1.568 GB 417.412 MB 26% 20%
<QUERY> 88.906K 7.917K 9% 3.346 GB 120.036 MB 4% 43%
Java, VBS & CSS 23.704K 18.564K 78% 164.808 MB 134.401 MB 82% 2%
/ -any content possible 17.124K 2.402K 14% 138.026 MB 12.741 MB 9% 2%
Web Pages: Static 10.335K 5.838K 56% 72.623 MB 11.909 MB 16% 1%
Web Pages: Dynamic 5.041K 99 2% 12.187 MB 251.792 KB 2% 0%
Media: Audio 4.946K 3.152K 64% 117.522 MB 30.179 MB 26% 1%
4.543K 771 17% 2.249 GB 6.600 MB 0% 29%
Software Updates 1.251K 1 0% 130.277 MB 1.062 MB 1% 2%
Archives & Executables 252 144 57% 60.839 MB 1.367 MB 2% 1%
Text & RTF Files 164 41 25% 497.592 KB 45.960 KB 9% 0%
==============================================================================
~~~ Objects Size Report ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
---------|------ R E Q U E S T S -------|----------- T R A F F I C -----------
SIZE | total | misses | hits |hit%| total | misses | hits |hit%
---------|--------|--------|--------|----|----------|----------|----------|----
0-0.1KB 9.024K 7.848K 1.176K 13% 942 B 942 B 0 B 0%
0.1-1.0KB 285.242K 85.594K 199.648K 70% 138.178 MB 34.336 MB 103.842 MB 75%
1-5 KB 73.101K 49.281K 23.820K 33% 191.150 MB 130.571 MB 60.579 MB 32%
5-10 KB 25.635K 17.231K 8.404K 33% 171.327 MB 114.483 MB 56.844 MB 33%
10-50 KB 31.154K 20.777K 10.377K 33% 670.167 MB 461.258 MB 208.909 MB 31%
50-100KB 6.170K 4.896K 1.274K 21% 418.392 MB 333.901 MB 84.492 MB 20%
100-500KB 4.477K 3.558K 919 21% 812.208 MB 655.973 MB 156.235 MB 19%
0.5-1.0MB 188 130 58 31% 122.032 MB 86.925 MB 35.107 MB 29%
1-5 MB 158 156 2 1% 405.781 MB 401.879 MB 3.902 MB 1%
5-10 MB 70 67 3 4% 474.746 MB 454.469 MB 20.277 MB 4%
10-50 MB 170 170 0 0% 3.054 GB 3.054 GB 0 B 0%
50-100MB 5 5 0 0% 374.959 MB 374.959 MB 0 B 0%
>100MB 7 7 0 0% 1.083 GB 1.083 GB 0 B 0%
===============================================================================
~~~ Squid Result Codes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Result Code | Requests | Traffic |
|------------------------|-----------|------------|
TCP_HIT 186.320K 553.716 MB
TCP_MEM_HIT 23.313K 62.323 MB
TCP_IMS_HIT 18.305K 5.677 MB
TCP_REFRESH_HIT 15.923K 104.257 MB
TCP_NEGATIVE_HIT 1.820K 4.214 MB
TOTAL HITS: 245.681K 730.187 MB
|------------------------|-----------|------------|
TCP_MISS 167.673K 6.997 GB
TCP_CLIENT_REFRESH_MISS 14.405K 54.055 MB
TCP_REFRESH_MISS 7.471K 63.893 MB
TCP_SWAPFAIL_MISS 171 1.249 MB
==================================================
TOTAL MISSES: 189.720K 7.114 GB
TOTAL HITS VOLUME: 56.43 % 9.11 %
|------------------------|-----------|------------|
TCP_DENIED 1.181M 1.519 GB
TCP_REFRESH_FAIL_HIT 41 464.793 KB
==================================================
~~~ HTTP Status Codes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
## | Description/Name | Requests | Traffic | Group |
----|-------------------------|-----------|------------|---------|
403 Forbidden 1.182M 1.520 GB DENY
200 OK 200.552K 7.677 GB OK
302 Moved Temporarily 169.757K 92.735 MB OTHER
304 Not Modified 44.755K 14.751 MB OK
000 UDP Traffic 8.650K 0 B UDP
404 Not Found 3.317K 6.709 MB ERROR
301 Moved Permanently 3.237K 1.510 MB OTHER
204 No Content 3.071K 770.258 KB OTHER
504 Gateway Timeout 827 1.032 MB TIMEOUT
206 Parital Content 522 35.385 MB OK
502 Bad Gateway 246 325.073 KB ERROR
500 Internal Server Error 161 388.460 KB ERROR
400 Bad Request 62 43.270 KB ERROR
503 Service Unavailable 20 9.532 KB ERROR
201 Created 8 2.438 KB OK
211 6 2.051 KB
303 See Other 2 2.219 KB OTHER
401 Unauthorized 1 307 B DENY
406 Not Acceptable 1 1.506 KB DENY
==============================|===========|============|========|
TOTALS: 1.617M 9.347 GB
Group | Requests %Vol | Traffic %Vol |
---------------|------------------------|-----------------------|
DENY 1.182M 73.08% 1.520 GB 16.26%
OK 245.837K 15.21% 7.726 GB 82.66%
OTHER 176.067K 10.89% 94.999 MB 0.99%
UDP 8.650K 0.54% 0 B 0.00%
ERROR 3.806K 0.24% 7.458 MB 0.08%
TIMEOUT 827 0.05% 1.032 MB 0.01%
===============================================================
~~~ Request Methods ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
---------------|------- R E Q U E S T S -----|-------- T R A F F I C ---------|
Method | %VOL | Requests | denied | %VOL | Traffic | denied |
---------------|--------|----------|---------|--------|------------|----------|
CONNECT 73.16% 1.183M 1.181M 16.42% 1.534 GB 1.519 GB
GET 26.05% 421.215K 21 83.38% 7.793 GB 32.220 KB
POST 0.78% 12.566K 0 0.21% 19.808 MB 0 B
HEAD 0.01% 116 0 0.00% 39.006 KB 0 B
OPTIONS 0.00% 1 0 0.00% 284 B 0 B
==============================================================================
~~~ Protocols & Ports (Requests) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Port | Protocol | Total %VOL | Misses Miss% | Hits Hit% | Deny Deny%|
------|-----------|---------------|--------------|--------------|-------------|
HTTP 1.615M 99.90% 188.134K 12% 245.681K 15% 1.181M 73%
443 HTTPS 1.565K 0.10% 1.565K 100% 0 0% 0 0%
8000 16 0.00% 0 0% 0 0% 16 100%
FTP 4 0.00% 4 100% 0 0% 0 0%
12000 3 0.00% 0 0% 0 0% 3 100%
3128 2 0.00% 0 0% 0 0% 2 100%
==================|===============|==============|==============|=============|
TOTALS: 1.617M 100% 189.703K 12% 245.681K 15% 1.181M 73%
~~~ Protocols & Ports (Traffic) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Port | Protocol | Total %VOL | Misses Miss%| Hits Hit% | Deny
------|-----------|-----------------|---------------|---------------|----------
HTTP 9.286 GB 99.36% 7.054 GB 76% 730.187 MB 8% 1.519 GB
FTP 46.127 MB 0.48% 46.127 MB 100% 0 B 0% 0 B
443 HTTPS 15.455 MB 0.16% 15.455 MB 100% 0 B 0% 0 B
8000 21.555 KB 0.00% 0 B 0% 0 B 0% 21.555 KB
12000 4.093 KB 0.00% 0 B 0% 0 B 0% 4.093 KB
3128 2.701 KB 0.00% 0 B 0% 0 B 0% 2.701 KB
==================|=================|===============|===============|==========
TOTALS: 9.346 GB 100% 7.114 GB 76% 730.187 MB 8% 1.519 GB
~~~ All Done. Enjoy Your Report. / SCALAR (C) 2004, http://scalar.risk.az ~~~